An Unsupervised Approach for Automatic Activity Recognition based on Hidden Markov Model Regression

1 An Unsupervised Approach for Automatic Acti vity Recognition b ased on Hidden Marko v Model Re g ression D. T rabelsi, S. Mohammed, F . Chamroukhi, L. Oukhellou, and Y . Amirat Abstract —Using supervised machine learning approaches to recog nize human activities from on-body wearable accelero meters generally r equires a large am ount of labelled data. When ground truth inf ormation is not a vailable, too expensive , time c onsuming or difficult to collect, one h as to rely on unsupervised appro aches. This p aper presents a new unsupervised approa ch for h uman activ ity recognition from raw acceleration data measured using inertial wearable sensors. The proposed method is based upon joint segme ntation of multidimen sional time ser ies using a Hid den Markov Model (HMM) i n a multiple regression context. The model is learned in an unsupervised framework usin g the Expectation- Maximization (EM ) algorithm where n o activity lab els are needed. The proposed method takes into account the sequ ential appearance of the data. It is theref ore adapted f or th e temporal acceleration data to accurately detect the a ctivities. It allows both segmentation and classification o f the human activities. Experimental results are pro vided to demonstrate t he efficien cy of the proposed approach wi th respect to standard supervised and unsup ervised classification approaches. Index T erms —Unsupervised learning, activity recog nition, wearable computi ng, mu ltiva riate reg ression, Hidd en M arko v Model I . I N T RO D U C T I O N T HE aging pop ulation has r ecently gained an increasing attention due to its socio-eco nomic impa ct. By 205 0 , the nu mber of p eople in th e Eur opean Union aged 65 and above is expected to grow by 70% and the n umber o f people a ged over 80 b y 170 % 1 . Th is dem ograph ic chan ge poses increasing challen ges for healthcare services and their adaptation to the needs of this ag ing pop ulation. Facing this problem or r educing its effect would h av e a gr eat societal impact by impr oving the q uality of life and regainin g peop le indepen dence to make them acti ve in society . The aim is therefor e to facilitate the daily activity li ves of elderly or depend ent people at home , to in crease their auton omy and to improve their safe ty . In fact, mo st elderly pref er to stay at home in the so-called “aging in place” [2]. The emergen ce of novel ad apted tec hnolog ies such as wear able and u biquitou s technolog ies is becomin g a privileged solution to provide D. Tra belsi, S. Mohammed and Y . Amirat are with Uni versit y Paris- Est Cr ´ eteil (UPEC), LISSI, 122 rue Paul Armangot, 94400, Vi try- Sur-Sei ne, France e-mail: dorra.tra belsi@etu .u-pec.fr , samer .mohammed@u- pec.fr , amirat@u-pec .fr . F . Cha mroukhi is with Uni versity Sud T oulon- V ar , LSIS Lab UMR CNRS 7296, e mail: Faicel .Chamroukhi@uni v-tln.fr . L. Oukhello u is with Uni versity Paris-Est, IFSTT AR, GRETTIA, F-9 3166 Noisy- le-Grand, Franc e, e-mail : latifa.ou khellou@i fsttar .fr 1 http:/ /ec.europa .eu/health-eu/my healt h/elderl y/ assisti ve services to hu mans, such as h ealth monitor ing, well being, security , etc. Among which, activity recognition has a wide range o f promising ap plications in secu rity monitorin g as we ll a s h uman mach ine interactio n [3]. A large am ount o f work has been d one in this active top ic over the past decad es; nevertheless it is still an open and ch allenging pro blem [4]. Sev eral technique s have been used to qu antify th ese activities such as video-b ased sensors [5], wearab le-based sensors, en viron mental sen sors a nd object sensors (smart p hones, RFID, etc.). Recen tly , the use of wearable-sensors based systems for activity recog nition has gained mo re attention on a lar ge number of techn ological fields such as navigation, monitorin g and control of aircrafts [6 ], [7], medical application [8], [ 9], loca lization and robo ts [10], [11]. Amo ng th e iner tial sensors u sed f or activity recogn ition, the accelerom eters are the m ost com monly used [12 ]. T hey have shown satisfactory results to m easure the h uman activities in both laborator y/clinical and free-living environment settings [13]. In addition, the latest advances in Micro-Electro Mechanical Systems (MEMS) tec hnolog y have gre atly pr omoted the use of accelerome ters thanks to the considera ble reduction in size, cost and energy con sumption . Early studies in acti vity recogn ition used un iaxial accelerometer s, while rece nt studies use mainly tri-axial accelerometer s [14], [15]. I I . R E L A T E D W O R K O N H U M A N A C T I V I T Y R E C O G N I T I O N Regarding the h uman activity classification, one can m ake the distinc tion b etween supervised an d u nsuperv ised classifi- cation appro aches. Supervised classification techniqu es consist in inferrin g a decision ru le from lab elled trainin g data. The use of th e superv ised a ctivity classification app roaches has shown pro mising re sults [16]. Some superv ised a pproa ches have enhanc ed the activity r ecognitio n proc ess pe rforma nces by using spatio-tem poral inform ation [17]. Regarding the algorithm s used in the s uperv ised context, on e can cite k - Nearest Neighb or ( k -NN) algo rithm [18], mu lti-class Suppor t V ector Ma chines (SVM) [19] and Artificial Neural Networks (ANN) including both MultiLayer Perceptron (MLP) [20], [21] and Radial Basis Function (RBF) network s [22]. Nev ertheless, the collection of suf ficient amounts of labelled data f or a various and ric h set of fr ee-living acti vities may be sometimes d ifficult to achiev e and computa tionally expen si ve [23]. On the o ther hand, un supervised classification techn iques try to d irectly construct models from un labelled data e ither 2 by estimating th e properties of their u nderlyin g prob ability density (called den sity estimation) or b y discovering gro ups of similar examples (called clusterin g). The unsup ervised learning techn iques are of particular interest for an exploratory analysis of large amounts o f unlab elled da ta. They can also consist in a preliminar y task to further run a supervised classifier b ased o n the obtained p artition of the d ata. T he use of an unsuper vised approach may be needed in such a context of activity r ecognition when it is d ifficult to have labe ls fo r the data. Regarding the app roaches used in th e unsuperv ised context, one can cite th e well-known k - Means algorithm [24], th e Gaussian Mixtur e Models (GMM ) appro ach [2 5] and the one based on Hidde n Mar kov Mod el (HMM ) [2 6], [2 7] or HMM with GMM em ission p robab ilities [28]. Both the GMM an d the HMM app roaches u se the EM algorithm [2 9]. The HMM has shown goo d r esults in earlier explorator y studies than ks to their main advantage of suitab ility to model sequential d ata which is the case of monitor ing human activities. Indeed, the acceler ation data are measured over time d uring ph ysical h uman activities of a person and are therefor e sequential over time. The EM algorithm [29] (also called Bau m-W elch [1]) in the con text of HMM is particularity adapted for unsupe rvised learning . In th is study , an un supervised app roach for hu man activity recogn ition is propo sed. It comb ines an HMM- based mod el with the use of acceleration data acquire d du ring sequenc es of different human activities. More specifically , the p roposed approa ch is based on a Hidden Markov Mod el in a multip le regression context an d will be denoted by MHMM R. As the sequences of acceleration data consist in multidimen- sional time series where each dimension is an acceleration, the activity recog nition problem is therefo re fo rmulated throug h the propo sed MHMMR model as the one of joint segmentation of m ultidimension al time series, each segment is associated with an activity . In the proposed mode l, each acti vity is represented by a regression model and the switching f rom one activity to an other is governed by a h idden Markov chain. The MHMMR parameters are learned in an unsup ervised w ay from unlab elled raw acceleratio n d ata acquir ed d uring human activities. The most likely seque nce of acti vities is then estimated using the V iterbi algorithm [3 0]. The pr oposed technique is then e valuated on real-world acceleration data co llected from three sen sors pla ced at the chest, the right thigh an d th e left ankle of the subject. This stud y is an exten sion o f th e pa per [31] where add itional technical im plementatio ns are shown: T welve activities and transitions are studied and perfor mances of the prop osed approa ch are evaluated and co mpared to tho se of some well known unsupe rvised an d s uperv ised techn iques for acti vity recogn ition. This paper is o rganized as fo llows; section III p resents the experimental proto col and the data acquisition platform. Sec- tion IV presents th e propo sed mo del and its un superv ised parameter estimation technique f rom u nlabelled acceleratio n data. In section V, the perfor mances of the prop osed app roach are evaluated and co mpared to th ose o f so me we ll known un- supervised and supe rvised tech niques fo r activity reco gnition. I I I . D AT A C O L L E C T I O N In this study , human activities are classified using three sensors placed at th e chest, the rig ht thigh and the left ankle respectively as shown in Figure 1. Sensor s p lacement is chosen to r epresent predomin antly upper-bod y activities such as stand ing up, sitting down, etc. an d pr edomin antly lower body activities such as walking, stair ascent, stair descen t, etc. The sensor ’ s placemen t guaran tees at the same time less constraint and better comfor t for the weare r . T he attachmen t of the sensors to the human body should be well fitted and secured (Fig. 1). These sen sors con sist of thr ee MTx 3- DOF inertial tr ackers developed by Xsens T echn ologies [3 2]. Each MTx unit consists of a tri-ax ial accelerom eter measuring the acceleration in the 3 -D space (with a d ynamic rang e of ± 5 g where g represents the gravitational constant). Our experiences show also tha t the measured ankle-sensor a ccelerations during the different activities do n ot exceed the limit o f ± 5 g. The sampling frequ ency is set to 25 Hz, which is sufficient and larger th an 20 Hz the required f requen cy to assess daily physical activity [33]. T he sen sors were fixed on the sub ject with the he lp of an assistant before the beginnin g of the measuremen t operatio n. Raw acc eleration d ata are therefo re collected over time when perform ing the activities. The MTx units ar e connected to a cen tral u nit called Xbus Master that is attached to the subject’ s b elt. Figure 2 shows the data gathering process from the Xbus-MTx acquisition system to the host pc. The Xbus Master is directly co nnected to the chest MT x unit while the remainig MTx units (thigh an d ankle) are connected in series. Data transmission b etween the Xbus Master and the pc is carr ied out throug h a Bluetooth wireless link. Fig. 1 . MTx-Xbus inerti al tracke r and sensors placement Fig. 2. Data ghatheri ng from the MT x-Xbus acqu isition system The experiments were perfor med at the LISSI Lab/University o f Paris-Est Cr ´ eteil (UPEC) by six different 3 healthy subjects of d ifferent ages (who are n ot th e researche rs) in the of fice environmen t. In or der to gather various and rich dataset, th e recru ited volunteer subjects have been chosen in a gi ven margin of age (25-3 0) and weig ht (55-70) kg. T welve activities an d transitions were studie d and are listed as f ollows: Stairs down (A 1 ) - Standing (A 2 ) - Sitting d own (A 3 ) - Sitting (A 4 ) - From sitting to sitting on the grou nd (A 5 ) - Sitting on the ground (A 6 ) - L y ing down (A 7 ) - L yin g (A 8 ) - From lyin g to sitting o n the g roun d (A 9 ) - Standing up (A 10 ) - W alking ( A 11 ) - Stairs up (A 1 2 ). Th e activities wer e chosen to have an approp riate representation of ev eryday activities inv olving different parts of the body (fig. 3). The recogn ized activities and transition differ in du ration and intensity level. No te that the activities A 3 , A 5 , A 7 , A 9 and A 10 represent dyn amic transitions b etween static acti vities. Each subje ct was asked to per form the twelve activities in his own style an d was not restricted on h ow the acti vities should be perfo rmed but only with the seque ntial activities order . In addition, th e dur ation of each activity is not re stricted to be the same as it may vary fr om one subjec t to another . Fig. 3. Examples of some considere d acti vities: a) Climbing Stairs Down, b) Climbing Stairs Up, c) W alking, d) Sitting, e) Standing Up, f) Sitting on the ground. W ith thr ee MTx sensor units, each one with a tr i-axial accelerometer, a to tal of nine a ccelerations are therefore measured an d reco rded overtime fo r each activity . Since the goal is to reco gnize human activities from on ly the raw acceleration data, the acquired acceleration signals can be seen as multid imensional time series (of d imension 9) with regime ch anges du e to the ch anges of activities over time. The activity reco gnition pro blem can ther efore be formulated as a problem of multidimensional time series segmentation. Ind eed, segmenting th e time series acco rding to different u nknown regimes over time is equiv alent to classifyin g the acc eleration data into o ne set of a ctivities; each activity being associated with a regime. T his will be detailed in the next section that is dedicated to the p roposed Hidden Markov Model Regression (HMMR) ap proach . I V . S E G M E N TA T I O N W I T H M U LT I P L E H I D D E N M A R K O V M O D E L R E G R E S S I O N - M H M M R In this section, the problem of activity rec ognition (clas- sification) is form ulated a s the one of join t segmen tation o f multidimen sional time serie s. Indee d, the acceleration data are presented as multidimension al time series p resenting various regime chang es. In such context, the g oal is to provide an au- tomatic pa rtition o f the data into different segments (regimes), each segmen t bein g consid ered after wards as an activity . V arious m odelling appro aches of time series presenting regime chang es have been p roposed in literature. O ne c an cite in particular the piecewise r egression as on e of the most adapted modelling approach es [34], [35]. The piecewise mo del has been applied in many domains including fina nce, engineer- ing, economics, and bioinform atics [36]. In the piecewise re- gression model [35], data are partitioned into se veral segments, each segment being character ized by its mean p olyno mial curve and its v ariance. Howe ver , the p arameter estimation in such method requ ires the use of dynamic progra mming algorithm [37], [3 8] which m ay be compu tationally expensive especially for time series with large nu mber o f ob servations. Moreover , the s tandar d piecewise regression mod el usually assumes that n oise variance is unif orm for all the segments (homo skedastic model). An alter native appro ach exten ded in this paper is based on Hidden Mar kov Model Regression [3 9]. This approa ch can be seen as an extension of the standard Hidden Markov Model (HMM) [ 27] to r egression analysis. Each regime is described by a regression model rather than a simple constan t m ean over time, while p reserving the Mar kov process modelling f or the sequen ce of unknown (hidden) activities. Indee d, standar d HMM-based approach es use simple Gaussian d ensities as density of ob servation. Howev er , in the HMM regression con text, each observation is assumed to be a noisy polyn omial func tion to better mo del very structu red data as the ac celeration d ata. The ap proach we pro pose fu rther extends the HMM model to a multiple regression setting . This is d ue to the fact the observed acceleration data is multidi- mensional. In the following, th e H idden Markov Regression Model for time series modelling is used by formulating its basic and multiple regression setting. In th is framework, each observation, den oted by y i , represen ts the i th acceler ation measuremen t while the associated state ( class), d enoted by z i , represents its correspo nding activity . A. General descriptio n of th e Multiple Hidden Markov Mo del Re gr ession In Hidden Markov Mo del Regression ( HMMR), each time series is represented as a sequence of ob served u niv ariate variables ( y 1 , y 2 , . . . , y n ) , wher e the observation y i at time t i is assumed to be g enerated by the fo llowing regression model [39]: y i = β T z i t i + σ z i ǫ i ; ǫ i ∼ N ( 0 , 1) , ( i = 1 , . . . , n ) (1) where z i ∈ { 1 , . . . , K } is a h idden discrete-valued variable. In this app lication case, z i represents the hidden class label (ac- ti vity) of each accelera tion data poin t and K corresp onds to the number of conside red activities. Th e variable z i controls the switching fro m o ne poly nomial regression mo del associated to o ne a ctivity , to ano ther o f K m odels at time t i . The vector β z i = ( β z i 0 , . . . , β z i p ) T is the o ne o f regression c oefficients of the p -order polynomial regression mod el z i and σ z i is its corresp onding standard deviation, t i = (1 , t i , t 2 i . . . , t p i ) T is th e p + 1 dimensional covariate vector at time t i and the ǫ i ’ s are stand ard Gaussian variables representin g an ad- ditiv e noise. The HMM R assumes that the h idden sequen ce z = ( z 1 , . . . , z n ) is a h omogen eous Markov chain of fir st 4 order p arameterize d by the initial state distribution π and the transition matrix A . It can be shown that, c ondition ally o n a regression mo del k ( z i = k ) , y i has a Gaussian d istribution with mean β T k t i and variance σ 2 k . Regarding the multiple regression case, the m odel can be form ulated as follows: y (1) i = β (1) T z i t i + σ (1) z i ǫ i y (2) i = β (2) T z i t i + σ (2) z i ǫ i . . . . . . y ( d ) i = β ( d ) T z i t i + σ ( d ) z i ǫ i (2) where d represents the dimensio n of the time series ( sequence) and the latent pro cess z simultaneou sly governs all the univ ari- ate time series compo nents. Th e m odel (2) can be rewritten in a matrix fo rm as follows: y i = B T z i t i + e i ; e i ∼ N ( 0 , Σ z i ) , ( i = 1 , . . . , n ) (3 ) where y i = ( y (1) i , . . . , y ( d ) i ) T is the i th observation of the time series in R d , B k = h β (1) k , . . . , β ( d ) k i is a ( p + 1) × d dimensiona l m atrix o f the multip le regression mo del p aram- eters associated with th e regime (class) z i = k and Σ z i its correspo nding covariance matrix. The Multiple HMMR mod el is ther efore fully parameteriz ed by the param eter vector θ = ( π , A , B 1 , . . . , B K , Σ 1 , . . . , Σ K ) . The next sub-section giv es the para meter estimatio n techn ique by maximizin g the o bserved data likeliho od throu gh the Expectation - Maximization (EM) algor ithm. Th e pa rameter vector θ is estimated using th e well-known maximum likelihood metho d thanks to its v ery well-known attra ctiv e limiting pro perties of consistency , asym ptotic normality a nd efficiency . In deed, in our experiments, a considerable num ber o f data points is acquired during time, which makes the samp le size suitable to take advantage of the limitin g p roperties of the maxim um likelihood estimator . The log -likelihood to be maximized in this case is written as fo llows: L ( θ ) = log p ( y 1 , . . . , y n ; θ ) = log X z p ( z 1 ; π ) n Y i =2 p ( z i | z i − 1 ; A ) n Y i =1 N ( y i ; B T z i t i , Σ z i ) (4) Since this lo g-likelihood cannot be maximized d irectly , th is can be perf ormed using th e EM algorith m [29], [40], th at is known as the Baum-W elch algorithm in the HMM context [1], [27]. T his algorith m altern ates between the two fo llowing steps: a) E-step: This step c omputes the conditional expecta- tion of the com plete-data log-likelihood giv en th e obser ved data Y , time t and a curren t p arameter estimation d enoted by θ ( q ) : Q ( θ , θ ( q ) ) = E h log p ( Y , z | t ; θ ) | Y , t ; θ ( q ) i · (5) It can b e easily sh own that this step only requ ires the calcu- lation o f: • the posterior probab ility τ ( q ) ik = p ( z i = k | Y , t ; θ ( q ) ) (6) ∀ i = 1 , . . . , n and k = 1 , . . . , K which is the po sterior probab ility that y i originates fro m the k th poly nomial regression m odel g iv en the whole ob servation sequen ce and the curr ent param eter estimation θ ( q ) , • and the join t po sterior prob ability of the state k at time i and the state ℓ at time i − 1 gi ven the wh ole obser vation sequence an d th e cu rrent parameter estimatio n θ ( q ) , that is ξ ( q ) iℓk = p ( z i = k , z i − 1 = ℓ | Y , t ; θ ( q ) ) (7) ∀ i = 2 , . . . , n and k , ℓ = 1 , . . . , K . These posterio r pro babilities ar e co mputed by the f orward- backward pro cedures in the same way a s fo r a standard HMM [27]. More calculation details on this step can be foun d in [2 7]. b) M-step: In this step , the value of the parameter θ is updated by computing the parameter θ ( q +1) that maximizes the conditional expecta tion (5) with respect to θ . It can be shown tha t this maximization lead s to the following updatin g rules. The up dates of the param eters governing the hidd en Markov chain z ar e the ones of a standard HMM and are giv en by : π ( q +1) k = τ ( q ) 1 k (8) A ( q +1) ℓk = P n i =2 ξ ( q ) ikℓ P n i =2 τ ( q ) ik (9) Updating the regression parameter consists in performin g K weighted m ultiple p olyno mial regressions. The r egression parameter matrices upd ates are giv en by : B ( q +1) k = h n X i =1 τ ( q ) ik t i t T i i − 1 n X i =1 τ ( q ) ik t i y T i = ( X T W ( q ) k X ) − 1 X T W ( q ) k Y , (10) where W ( q ) k is a n × n diag onal matr ix of weig hts whose di- agonal elemen ts are the posterior p robab ilities ( τ ( q ) 1 k , . . . , τ ( q ) nk ) and X is the n × ( p + 1) regression matrix given by ( t 1 , . . . , t n ) T . The u pdating rule fo r the covariance matrices is written as a weigh ted v a riant of the estimation of a mu ltiv ar iate Gaussian den sity with the polyno mial me an B T ( q +1) k t i such as: Σ ( q +1) k = 1 P n i =1 τ ( q ) ik n X i =1 τ ( q ) ik ( y i − B T ( q +1) k t i ) T ( y i − B T ( q +1) k t i ) = 1 P n i =1 τ ( q ) ik ( Y − XB ( q +1) k ) T W ( q ) k ( Y − X B ( q +1) k ) (11) V . R E S U LT S A N D D I S C U S S I O N S This section presen ts experim ents carrie d ou t to v alidate the two main ideas explored throug hout this p aper, i.e., the segmentation and th e classification of th e h uman activity from raw accelera tion da ta u sing a MHM MR appro ach w ithin an unsuperv ised learn ing f ramework 2 . Series of experiments were co nducted to evaluate the pe rforma nce of th e propo sed approa ch an d also to per form compa risons with well-known unsuper vised and supervised classification appro aches. 2 Note that, in this study , the raw accelerati on data are direct ly used without any feature extract ion. Indeed, in many area of applicati on a feature extracti on step is needed before running the classifier and may itself lead to an addition al computat ional cost, w hich can be penali zing in real time applicatio ns. 5 A. P erforma nce evalu ation Giv en a set of 9-dimen sional accelera tion data from three triaxial accelerome ter mod ules mo unted on the chest, right thigh and le ft ank le, the prop osed approa ch allows both segmentation and classification o f the twelve activities. Each obtained segment is ind eed considered as an acti vity , achie ving thus a classification task. W e cho se to take as a groun d truth about the class of an acti v ity , labeling obtained thanks to an exper t. While the different subjects were perf orming the sequence of acti vities, an independent operator was asked to annotate th e acti v ities, thus pr oviding a labeling o f the dataset 3 . The provided partition is indeed matched to the true labels (grou nd truth) by ev aluating all the possible lab el switchings. Th e label switchin g leading to the minim um error rate is selected as th e b est class pr ediction. For the supe rvised classification ap proach es, data labels were used to both train and test the models. In this case, the perform ance was esti- mated throug h a 10 -fold cross-validation procedure. Regardin g the classification pr oblem, confu sion matrices between the annotated classes an d the estimated classes for all the subjects in the database are compu ted. The c riteria used to evaluate the perfor mance o f an appro ach are th e cor rect classification rate and the pred iction accu racy in terms of precision and recall. In the f ollowing, the results o f the MHM MR app roach obtained on re al acceleration d ata of h uman activities are first detailed , then they are compared to those o f standard unsuper vised and supervised classification appro aches. B. Classification performance o f the MHMMR The f ollowing experim ents wer e conducted to qualitatively assess the perfor mances of the p roposed approa ch in ter ms of automa tic segmentation of h uman activity on the basis of raw ac celeration sig nals. From th e sequenc e of nin e observed variables y i = ( y (1) i , . . . , y (9) i ) at each time step i for i = 1 , . . . , n corr espondin g to the 3-a xis acceler ations measure d by the three sen sors, the M HMMR is used to identify the latent sequence z = ( z 1 , . . . , z n ) correspon ding to the twelve activities. T he nu mber of cla sses K is fixed to twelve an d the order of regression p is fixed empirically to three as it gi ves the best performa nce among se veral values of p . Model parameters are estimated from the data using the algorithm d etailed in Section IV -A. Figures 4 and 5 show the p erform ance of the propo sed m ethod to segment the two following sequences: • Sequ ence 1: Stan ding - Sitting d own - Sitting - Fro m sitting to sitting on the ground - Sitting on the g roun d - L ying down - L y ing, • Sequ ence 2 : Stan ding - W alking - Climbing up stair s - Standing. These figures repr esent the ev o lution of the acceleration data and the cor respond ing posterior pro babilities for the two different sequences. Note that th e po sterior prob ability is the probab ility tha t a sample i will be generated by the regression model k given the wh ole sequen ce o f observations ( y 1 , . . . , y n ) . I t can b e ob served that the o btained seq uences 3 Note that the labels were not used to train unsupervised models; they were only used afterward s for the ev aluat ion of classificat ion errors. -15 -10 -5 0 5 10 15 0 5 10 15 20 25 1 0 k = 1 k = 2 k = 3 k = 4 k = 7 k = 6 k = 5 T i m e ( s ) A c c e l e r a t i o n ( m / s 2 ) P r o b a b i l i t y T r u e L a b e l s A 2 A 3 A 4 A 5 A 6 A 7 A 8 Fig. 4. MHMRM segmentatio n for the sequence (Standing A 2 - Sitting down A 3 - Sitting A 4 - From sitting to sitting on the ground A 5 - Sitting on the ground A 6 - L ying do wn A 7 - L ying A 8 ) for the sev en classes k=(1,. . . , 7) -10 -5 0 5 10 15 50 55 60 0 1 k = 1 k = 2 k = 3 k = 1 T i m e (s ) A 2 A 1 1 A 1 2 A 2 A cc eler at io n ( m / s 2 ) P ro bab ilit y Tr ue Label s Fig. 5. MHMMR segmenta tion for the sequence (Standing A 2 - W alking A 11 - Climbing up stairs A 12 - Standing A 2 ) for the three classes k= 1,. . . , 3 are intere sting and prom ising d espite some con fusion between activities such as (A 11 , A 12 ). T able II shows that the MHMM R gives 91 . 4% as a m ean correct classification ra te averaged over all observations. I t highligh ts the potential ben efit of the proposed ap proach in terms of auto matic segmentation and c lassification of hum an activity . Both the transitions and the stationary acti vities are well identified. Exh austively , table I gives th e percen tage of precision and recall for eac h ac ti vity . Indeed, one can observe that static activities (A 2 , A 4 , A 6 a nd A 7 ) ar e easier to recog nize than d ynamic activities ( A 1 , A 11 , A 12 ). I n ord er to focu s on the efficiency ratio of the three sensors used for activity recognition , the MHMMR algo rithm has bee n ev a luated u sing data from only two senso rs. T he classification results, giv en in T able II, show as expec ted, that the percentage of corr ectly classified instances decreases with the n umber of data sources. The worst r esult is ob tained when the sen sor placed at the thigh is no t taken into account. 6 Class A 1 A 2 A 3 A 4 A 5 A 6 A 7 A 8 A 9 A 10 A 11 A 12 Precision ( % ) 71.9 96.4 78.4 95.7 92.3 98.9 97.6 92.5 82.6 82.6 83.2 95.6 Recal l ( % ) 95 87.8 83.4 94 97.3 94.6 95.4 90.9 98.5 92.2 98.1 82.3 T ABL E I R E C A L L V E R S U S P R E C I S I O N O F T H E M H M M R Sensors Percent age of correctly classified instances Chest, thigh, ankle 91 . 4% ± 1 . 65 Chest, ankle 83 . 9% ± 1 . 98 Chest, thigh 86 . 2% ± 2 . 03 Thigh, ankle 84% ± 2 . 21 T ABL E II E FF E C T S O F R E D U C I N G T H E N U M B E R O F S E N S O R S W H E N U S I N G M H M M R C. Compa rison with unsup ervised an d supervised classifica- tion a ppr oaches Correct classification rates and the standar d deviations ob - tained with stand ard unsuper vised an d superv ised classifica- tion ap proach es as well as the MHMMR app roach are given in T able III and T able IV. Comp ared to stand ard u nsuperv ised Correct Classification ( % ) Precision ( % ) Recall ( % ) k -Means 60 . 2 ± 2 . 48 60 . 4 59 . 8 GMM 72 . 3 ± 2 . 05 71 . 8 73 . 5 HMM 84 . 1 ± 1 . 84 83 . 8 84 MHMMR 91 . 4 ± 1 . 65 89 95 . 6 T ABL E III C O M PA R I S O N O F T H E P E R F O R M A N C E I N T E R M S O F C O R R E C T C L A S S I FI C A T I O N , R E C A L L A N D P R E C I S I O N O F T H E F O U R U N S U P E RV I S E D C L A S S I FI E R S classifiers, the prop osed M HMMR outper forms them since it provides a classification rate of 91.4 % wh ile o nly 60 % , 72% and 84% of instances are well classified with r espectively the k -Mean s, the GMM and the standa rd HM M approa ches. Notice that, the GMM and K-m eans approach es are n ot well suitable for this kin d of longitudinal data. In T able IV, it can be Correct Classificat ion Precision Recall ( % ) ( % ) ( % ) Nai ve Bayes 80 . 6 ± 0 . 91 80 . 9 80 . 6 MLP 83 . 1 ± 0 . 45 82 . 8 83 . 2 SVM 88 . 1 ± 1 . 32 87 . 6 88 . 3 k -NN 95 . 8 ± 0 . 32 95 . 9 95 . 9 Random Forest 93 . 5 ± 0 . 78 93 . 5 93 . 5 T ABL E IV C O M PA R I S O N O F T H E P E R F O R M A N C E I N T E R M S O F C O R R E C T C L A S S I FI C A T I O N , R E C A L L A N D P R E C I S I O N O F T H E FI V E S U P E RV I S E D C L A S S I FI E R S observed that the k -NN ( k = 1 ) g iv es the high est classification rates with 95.8 % , followed by th e Rand om Forest with 93 .5 % . Then, the SVM g iv es 88.1 % and th e MLP gi ves 8 3.1 % . Howe ver , the Naiv e Bayes gives the lowest classification rate with 80.6 % . T able IV shows also that the k - NN ( k = 1 ) has the best classification algorithm in ter ms of prediction a ccuracy since it ac hieves 95.9 % of p recision and recall. Compared to standar d supervised classification techniqu es ( using class labels), th ese resu lts are very encour aging sinc e the propo sed approa ch perfo rms in an unsup ervised way . The main er rors are due to the con fusions located in transition segments. This is d ue to th e fact th at th e transitions lengths are muc h shorter than th e activities ones. Since the co nfusion matrix was comp uted using rea l labels supplied b y a human expert, the obtained labels may no t co rrespon d p erfectly to the exper t labels, particu larly , d uring transition s. In deed, it is difficult to have the g round truth of the limit b etween an activity and a transition. Furthermore, the afor ecited s uper vised class ification approa ches r equire a lab elled collectio n of d ata to be tra ined. Besides, they d o no t explicit th e tempor al depe ndence in their model fo rmulation as they assum e an independen t hypo thesis for the data; the data are tr eated as several r ealizations in the multidimensional spa ce ( R d ) without considering pos- sible depen dencies between the activities. Moreover , it can be noticed that assignin g a ne w s ample to a class u sing the k -NN approach requires the computation o f as many distances as there ar e examples in the d ataset, which may lead to a sign ificant co mputation time. Using the pro posed approa ch, classification need s the compu tation of the p osterior probab ilities, as many as th ere are activities. On the other han d, compariso n with the unsup ervised classification ap proach es ( k -Mean s and the GMM) and the standard HMM shows th at the proposed m ethod g iv es relatively a high rate and better perfor mances. V I . C O N C L U S I O N A N D F U T U R E W O R K S In this pape r , we presented a statistical a pproac h b ased on hidden Markov mo dels in a regression con text for the joint segmentation of multiv ariate time series of h uman activities. It is based up on the use of raw accelerometer data acquired from bod y mounted in ertial sensors in a health-mo nitoring context. The main advantage of the pr oposed app roach comes from the fact th at the statistical m odel explains the regime changes over time through th e hidd en Mar kov chain , each regime being interpr eted as an activity (a class). Furthermo re, learning with th is statistical m odel is performed in an un - supervised w ay using un labelled examp les only; parameter estimates are computed by maximizin g a likelihood criter ion, using a dedicated EM algorithm. Considering human activity recogn ition with in an unsuper vised lea rning fr amew ork can be particularly in teresting with in an exp loratory data-m ining context in order to au tomatically cluster a large amo unt of unlabelled acceleratio n data in to different gr oups of activity . The compar ison with well-known sup ervised classification approa ches shows that th e pro posed m ethod is com petitiv e ev en when perform ed in an unsuperv ised way . Th is work can be extended in several directions, n amely integrating the model into a Bayesian co ntext to be tter contro l the m odel complexity via choosing suitable prior d istributions on the models paramete rs. Then, and perh aps more interestingly , 7 another step to explor e is to built a fu lly n on Bayesian no n- parametric model which will be useful for any kind of co mplex activities and in which the number o f activities will not h ave to be fixed. In terms of application , a promising p erspective in a reh abilitation context would be to use th e pro posed appro ach for recog nizing in an unsup ervised fram ew ork th e u ndesirab le compen satory p hysical beh aviours ob served with stroke an d injury p atients. R E F E R E N C E S [1] Baum, L .E. and Petrie, T . and Soules, G. and W eiss, N. , ”A maximizat ion techni que occurring in the statisti cal analysis of probabilist ic functions”, Annals of Mathematical Statisti cs , vol. 41, pp 164-171, 1970. [2] Kaluza, B., Mirche vska, V ., Dovgan, E., Lustrek, M., Gams M., ”An Agent-ba sed Approach to Care in Indep endent Li ving”, In L ecture Notes in Computer Science, Internati onal Joint Conferenc e on Ambient Intell igence (AmI-2010), Malaga, Spain, 6439, 177–186, 2010. [3] Lu, C.-H. and F u L-C., ”Robust Location -A ware Activi ty Recognit ion Using Wi reless Sensor Network in an Attenti ve Home”, IEEE Transac- tions on Automation Science and E ngineeri ng, V ol.6, No.4, pp.598-609, 2009. [4] Jinhui, Hu., and Niko laos, V .B, ”Fast human acti vity recognition based on structure and motion ”, Pattern Recognit ion Letters, 32(14), 1814–1821, 2011. [5] Brdiczka O., Langet M., Maisonna sse J. , Cro wley J. L., ”Detect ing human beha vior models from multimodal observ ation in a sm art home”, IEEE Tra nsactions on Automation Science and Engineeri ng, V ol . 6, No.4, pp 588-597, 2009. [6] MacK enzie, D. , ”Inv entin g accurac y A Historic al sociology of nuclea r missile guidance ”, MIT Press, Cambridge, MA, and London, 1991. [7] Carminati, M., Ferrari, G., Sampietro, M., Grassett i, R. , ”Fault detection and isolation enhancemen t of an aircraft attit ude and heading reference system based on MEMS inertial sensor”, Procedia Chemistry 1 (1), 509– 512, 2009. [8] Jov anov , E., Milenko vic, A., Otto, C., DeGroen P .C., ”A wireless body area network of intellige nt motion sensors for computer assisted physical rehabil itatio n”, J. Neuro Eng.Rehab . 2(6), 2005. [9] Wu, W . H., Bui, A. A. T ., Batalin, M. A., Liu, D., Kaiser , W . J., ”Incremen tal diagnosis method for intelli gent wearable sensor system”, IEEE Trans. Inf. T echnol.B 11 (5), 553-562, 2007. [10] Barshan, B., Durrant-Whyte, H.F ., ”Ev aluati on of a solid-sta te gyroscope for robot ics applicat ions”, IE EE T rans. Instru m. Meas. 44(1), 61-6 7, 1995. [11] T an, C.W ., Park, S., ”Design of acceleromet er-base d inertial navigatio n systems, IEEE Trans. Instrum. Meas. 54(6), 2520-2530, 2005. [12] Jiayang, L., Lin, Z., Wic kramasuriya, J. and V asudev an, V ., ”uwa ve : Accele rometer -based personalize d gesture recognition and its applica - tions”, Perv asi ve and Mobile Computing, 657–675, 2009. [13] Mathie, M. J., Celler , B. G., Lovell , N. H., Coster , A. C. F . , ”Classifi- catio n of basic daily movemen ts using a triaxal acceleromet er”, Medical and Biologic al E ngineeri ng and Computing, 42, 679–687, 2004. [14] Noury , N., Barralon, P . , V irone, G., Boissy , P ., Hamel, M. , Rumeau, P ., ”A sm art sensor based on rules and its ev aluat ion in daily routines”, In Procee dings of the 25th Annual Internation al Conference of the IE EE Engineeri ng in Medici ne and Biology Society , (EMBS), Cancun, Mexico, 3286–3289, 2003. [15] Y ang, J. Y ., W ang, J. S., Chen, Y . P . , ”Using accelerat ion measurements for acti vity recogniti on: An ef fecti ve learning algorithm for constructing neural classi fiers”, Pattern Recogn ition L etters, 29 (1 6), 2213–2220, 2008. [16] Altun, K ., Barshan, B., Tun cel, O. , ”Comparati ve study on classifying human acti vities with miniature inerti al and magneti c sensors”, Pattern Recogni tion, 43 (10), 3605–3620, 2010. [17] Khalili, A. H., and Aghajan, Ha., ”Multi vie w acti vity recognition in smart homes with spatio-t emporal features”, In Procee dings of the 4th A CM/IEEE ICDSC, 142–149,USA, 2010. [18] Liu, C. L., Lee, C. H., and Lin P . M., ”A fall detection system using k- nearest neighbor classifie r”, E xpert Systems with Applications, 37, 7174– 7181, 2010. [19] Qian, H., Mao, Y ., Xiang, W ., W ang, Z., ”Recogn ition o f human acti vities using SVM multi-class classifier”, Pattern Recognit ion Letters, 31 (2), 100–111, 2010. [20] Dong, L . , Che, L. , Sun, L., W ang, Y ., ”Effec ts of non-paralle l combs on reli able operati on conditions of capacit i ve inertial sensor for step and shock signals”, Sensors and Actuators A: Physical , 121 (2), 395–404, 2005. [21] Y ang, J. Y ., W ang, J. S., and Chen, Y . P ., ”Using accele ration mea- surements for acti vity recogniti on: An ef fecti ve learning algorithm for construct ing neural classifie rs”, Pattern Recognitio n Letters, 29, 342–350, 2008. [22] Faber , G. S., Kingma, I., Bruijn, S. M. , V an Dieen, J. H., ”Optimal inerti al sensor location for ambulatory measurement of trunk inclina tion”, Journal of Biomechanic s, 42 (14), 2406–2409, 2009. [23] Cvetk ovi c, B., Lustrek, M., Kaluza, B., and Gams, M., ”Semi-supervised Learning for Adaptation of Human Acti vity Recognition Classifier to the User”, Internatio nal Joint Conference on artificial Intelli gence (ST AMI), pp.24–29, 2011. [24] Duda, R. O. , Hart, P . E., and Stork, D. G., Patte rn Classification (second ed.) A Wile y-Interscience Publicatio n, John Wile y & Sons, 2000. [25] Allen, F . R. , Ambikairaja h, E., Lov ell, N. H. , and Cell er , B. G., ”Classific ation of a kno wn sequence of motions and postures from accel erometry data using adapte d Gaussian mixture models”, Physiol. Meas., 27(10), 935–951, 2006. [26] Lin, J. F . S., and Kul ic, D., ”Automatic human motion segmenta tion and identificat ion using feat ure guided hmm for physic al rehabili tation ex ercises” , In: Robotics for Neurology and Rehabi litati on, W orkshop at IEEE/RSJ Internati onal Conference on Intell igent Robots and Systems (IR OS), 2011. [27] Rabiner , L. R., A tutorial on hidden Marko v m odels and selecte d applic ations in speec h recogni tion, Proceedings of the IE EE, 77(2), 257– 286, 1989. [28] Mannini, A., and Sabati ni, A. , ”Machi ne learnin g m ethods for classi fy- ing human physical acti vity from on-body accel erometers”, Sensors, 10, 1154–1175, 2010. [29] Dempster , A. P ., L aird, N. M. , and Rubin, D. B. , ”Maximum lik elihood from incomplete data via the EM algorithm”, Journal of The Roya l Statist ical Society , B, 39(1), pp.1-38, 1977. [30] V iterbi, A. J., ”Error bounds for con volution al codes and an asymptot- icall y optimum decoding algorithm” , IEEE Transacti ons on Information Theory 13 (2), 260–269, 1967. [31] Trabe lsi D., Mohammed S., Oukhell ou L., Amirat Y ., ”Acti vity Recog- nition Using Body Mounted Sensors: An Unsupervised Learning based Approach” , IE EE W orld Congre ss on Computational Intelli gence - In- ternat ional Joint Conference on Neural Network s (IJCNN), June 10-15, Brisbane , Australia, 2012. [32] http:// www .xsens.com [33] Bouten, C., Koekk oek, K., V erduin, M., Kodde, R., Janssen, J., ”A triaxi al acceleromete r and portable data processing unit for the assessment of daily physical acti vity”, IEEE Tra nsactions on Biomedic al Engineering 44 (3), 136-47, 1997. [34] McGee, V . E., Carleton, W . T ., ”Piece w ise re gression”, Journal of the American Statistic al Ass ociati on 65, 1109–1124, 1970. [35] Brailo vsky , V . L., Kempner , Y ., ”Applic ation of piec e wise regre ssion to detec ting internal structure of signal”, Pattern recog nition 25 (11), 1361– 1370, 1992. [36] Picard, F ., Robin, S., Lebarbier , E. , and Daudin, J. J., ”A Segmenta- tion/C lustering Model for the Analysis of Array CGH Data”, Biomet rics, 63(3), 758–766, 2007. [37] Bellman, R., ”On the approximati on of curves by line segments using dynamic programmin g”, Communicati ons of the Ass ociat ion for Com- puting Machinery (CACM) 4 (6), 284, 1961. [38] Stone, H. , ”Approximation of curves by line segments”, Mathematics of Computati on 15 (73), 40–47, 1961. [39] Fridman, M., Hidden markov m odel regression, T ech. rep., Institute of mathemati cs, Univ ersity of Minnesota, 1993. [40] McLachlan, G. J., Krishnan, T ., T he EM algorit hm and extensions, Ne w Y ork: Wil ey , 1997. [41] Kunche va, L. I., Combining P attern Classifiers: Methods and Algo- rithms, W ile y-Interscie nce, 2004.