Implementation of Artifact Detection in Critical Care: A Methodological Review

RBME- 00025 -2012.R1 1 Abstrac t - Artifact Detection (AD) techniques minimize the impact of artifacts on physiolog ic data acquired in Critical Care Units (CCU) by assessing quality of data prior to Clinical E vent Detection (CED) and Pa rameter Derivation (PD). This methodological review introduces unique taxono m ies to synthesize over 80 AD algorithms based on these six theme s: (1) CCU; (2) Physiologic Data Source; (3 ) Harvested data; (4) Data Analysis; (5) Clinical Evaluation; and (6 ) Clinical Implementation. Review results show that most published algorithms: (a) are designed for one specific type of C CU; (b) are validated on data harvested on ly from one OEM monitor; (c) generate Sig nal Q uality Indicators (S QI) that are not yet formalised for useful int egration in clinical workflows; (d) operate either in standalone mode or coupled w ith CED or PD applications (e) are rarely evaluated in real-ti me; and (f) are not implemented in clinical practice. In conclusion, it is recommended that AD algorithms conform to generic input and output interfaces with commonly defined data: (1) type; (2 ) frequency; (3) length; an d (4) SQIs. This shall promote (a) reusability of algorithms acr oss diff erent CCU do mains; (b) evaluation on different OEM monitor data; (c) fair comparison through formalised SQIs; (d) meaningful integration w ith other AD, CED and PD algorith ms; and (e) real-ti me i mplementa tion in clinical workflows. I. I NTRODUCTI ON hysiologic signals exhibit trends, dynamics a nd correlations reflecting the complexity of underlying patient physiology [1, 2]. Continuous monitorin g of physiologic data assists cli nicians in making d iagnoses and prognoses in Critical Care Units (CCU), including Intensive Care (ICU), Paediatric I ntensive Care (P ICU), Neonatal Intensive Care Units (NICU), and the Operating Roo m (O R) [3]. Clinical Eve nt Detec tion (CED) techniques analyze these data to identif y cl inicall y si gnificant e vents and earl y onset indicators of vario us pathoph ysiologies as in [4 - 16] . Parameter Derivation (PD) techniq ues derive clinicall y useful lo w frequency p arameters from high frequency input data as in [17- 23] . Artifacts are extraneous signals with randomly varying amplitudes, frequencies and duratio n that i nterfere with physiologic signals acquired in clinical setti ngs [24] . Longitudinal studies [ 25- 28] infer that Origi nal Equipment Manufacturer (OEM) patient monitors have relativel y simplistic built-in data preprocessing for Artifact Detection (AD). In this paper, the term AD encompasse s one or more of the following m echanis ms: (a) id entification, detection or annotation o f arti facts, (b) eli mination o f artifact s or artifact - laden data and ( c) s uppression or filtering of arti facts e.g., by using ensemble averaging or ad aptive filtering [2 9]. The terms noise and clean ing have been used in p ublications as alternatives for the ter ms artifacts and AD respec tively. OE M monitors typicall y co me with a b lack box approac h to preprocessing artifacts i n p hysiologic data. Clinicians debate the reliability o f OEM monitor data as they ded uce that so me built-in al gorithms employ a reductionist approach that oversimplifies co mplex human physiolog y [30 ]. Artifacts can mimic p hysiologic data [31- 33] , adding to the chal lenge of distinguishing bet ween the two. As a result, data log ged b y OEM monitors re mains impacted by artifacts [34- 36] . This increases false alarm rate s i n m onitors [36- 38] , which lead s to staff desensitization [30, 39, 40 ]. Clinicians ca nnot rely on analysing artifact-laden monitor data [41], which in the past has resulted in incorrect diagnoses [33, 42]; unnecessary therapy; surgery and iatro genic diseases [32 ]. Independent research groups have developed a variety o f post pro cessing techniques to add ress the p roble m of artifact s in OEM monitor data. Fig. 1 shows the AD configurations reviewed in t his paper. T he purpose of AD is identical in any configuratio n, i.e., to assess a nd enhance the quality of ph ysiologic data. F ig. 1 depicts Signal Q uality Indicator s ( SQI) among ot her output variables. Latest AD research shows i ncreasing interest in devising SQIs [43- 46] . This p aper co nducts a critical review of the development and util ity of SQIs. T his methodolo gical revie w introduce s six thematic taxonomies for a critical a nalysis of t he state - of -the-art in AD. The o bjective is to synt hesize the status of clinical evaluatio n and implementation of AD in vario us CCUs. Reviewed publications are listed in Table I . Section  II discusses past reviews of AD. Section  III briefly d escribes research methodology followed by de tailed develop ment of a unique taxonomy for each of the six themes. Revie w re sults a re thematically synthesized in Table II . Section  IV concludes the review b y highlighting o pen research p roble ms. I t also provides specific recommendations for new research directions in promoting implementation of AD in real -time clinical workflows. II . R ELAT ED W ORK Historically, AD has been reviewed with significantly different scope and context fro m t his paper, wh ich makes t h is review a novel co ntribution in this research spac e. Boro wski et al. [47] have conducted a co mprehensive revie w on alar m generation in medical devices. T heir pr imary focus is Implementation of Artifact Detection in Critical Care: A Methodologi cal Review Shermeen Nizam i 1 , Student Me mber I EEE; James R. Green 1 , SMIEEE; Car oly n McGregor 2 , SMIEEE 1 Department of Systems and Computer Engineeri ng, Carleton Universit y, Canada 2 Faculties of Business and IT , and Health Sciences, University of Osha wa Institute of Technolog y, Canada P RBME- 00025 -2012.R1 2 ergonomics and human facto rs engineerin g, although, the y briefly synthesize some A D algorithms and, as a result, recommend using multivariat e alarm systems. Schettlin ger et al. [48] have largely revie wed t heir own research in developing univariate filters for outlier, trend and level shift detection in variou s ICU data t ypes. The y extensivel y descri be filter develop ment and compare advantages and disadvanta ges of different filter des igns. Cha mbrin [49] infers that multivariate tech niques can reduce false alarm rate s in ICUs. Numerous p ublications, such as [ 37, 43, 48, 50- 52] , provide comparative numeric sum maries of perfor mance metrics of AD a lgorit hms. This re view neither reiterates al gorithmic details nor performance comparison as that falls o utside its scope. T akla et a l. [28] note that while AD techniques developed by independent researchers may have highe r specificity than b uilt-in algorithms in OE M monitors, extensive studies ar e required to evaluate their accuracy prior to real -time impleme ntation in clinical e nvironments. Siebig et al. [38] dem onstrate agree ment amongst cli nical staff, including intensivists, t hat integrative monitoring t hrough d ata fusion ca n po tentially yield better results as co mpared to simpler univariate thres hold detectio n methods. I mhoff et al. [37, 52] e mphasize that m ethod ological resear ch is needed for integrating multivariate AD algorithms in real -time clini cal systems. However, they do not present a co nceptual framework to conduct such re search. III . M ETHOD OLOGICAL R EVIEW Papers of interest were locate d in Scopus, IE EE Xplo re and PubMed d atabases using t he keywords artifact, artefact, artifact detectio n, alarm, p hysiologic m onitoring, inte nsive care, neonatal intensive ca re, operating room and patient monitors a mongst ot hers. No constrai nts were app lied o n the publication year. Google Scholar was also used. Liter ature was t hen methodologically reviewed using six thematic taxonomies de veloped in this section: (1) Critical Care U nit, (2) Physiologic Data Source , (3) Harvested data, (4) Data Analysis, (5) Clinical Evaluation and (6) Clinical Implementation. Fig. 2 depicts an o verview of t he thema tic taxonomies. A. Critical Care Un it This theme catalo gues do main specif ic AD research to examine its reusabil ity and po rtability across other CCUs. The taxonomy d eveloped for this theme is: I CU, PICU, NI CU, OR and Other relevant studies. This ta xonomy, tabulated in Tab le I, reveals that al most all tech niques are evaluated in a si ngle domain, with the e xception of [23] which is evaluated i n t he ICU, PICU a nd OR. In general, trends in ph ysiologic sign als display similar dynamics ac ross CCU d omains [53] . This drives the hypothesis that AD algorithms could be modified for use acr oss di fferent cr itical care settings. Ho wever, do main specific infor mation suc h as types and frequency o f data, and patient de mographics s uch as range of a ge, weight and medical condition may be directly, or indirectl y, hard cod ed in the algorith m. Therefore, it is important to consider the original d omain of clinical ap plication prior to atte mpting application elsewhere. B. Physiolog ic Data Source This theme synthesizes the methodologies used to source physiologic data. The theme taxo nomy is: P hysiologic Monitor, Inclusion/Exclusion Criter ia and Sa mple Size, as tabulated in the second colu mn of T able II . T he types, frequency, numeric va lues a nd qualit y o f data p roduced by patient monitors (and pro bes) differs between models fro m the same o r different OEMs. This is due to d ifferent built -i n proprietar y signal pr eprocessing, i nclusive o f AD [28, 54] . For example, comparative studies show characteristic discrepancies i n neonatal data acq uired usin g v arious OE M Pulse Ox imeters (PO) including Masi mo SET Rad ical (Masimo Corp ., Irvine, C A, USA), Date x -Ohmeda TruSat (GE Hea lthcare, C halfont St Giles, UK), S i emens SC7000 (Siemens UK, Frimley, UK) , Nonin 7500 (Nonin Medic al, Plymouth, MN, US A), [55] ; Nellcor OxiMa x N-600x (Covidien-Nellco r, Boulder, CO, USA) [ 55, 56] ; Philips FAST MP50 [56] and Philips I ntellivue MP7 0 (P hilips, Ger many) [57]. Therefore, knowledge of the monitor OEM and m odel is important f or recognizing di fferent biases in troduced in the data. Some studies may collect data under certain inclusion and exclusion criteria a s shown in Table II . Sample size is almost always given in the literature. T he discretion generally lies with t he resear cher to determine if enough data is available to d raw a statistica lly and cli nically sound decision. Sample size may be deduced based o n tr ial, availability of resources and stud y requirements. C. Harvested Data The taxonomy de veloped under this the me i s: Data T ype and Acquisition/Sampling or Storage Freq uency. In T able II , the top half of each ce ll in column t hree su mmarizes the t ypes and frequencies of data harves ted in each stud y. Kno wledge o f the OEM is also a convenient indicato r o f this information. Routinely harvested or monitored physiologic data types in cr itical care include: Electrocardiograp h (ECG), Hea rt Rate (HR), Brea thing or Re spiratory Rate (RR), I mpedance Respiratory W ave (IRW), Noninvasive Oxygen Saturation (SpO 2 ), Invasive or no ninvasive A rterial O xygen Sa turation (SaO 2 ), Temperature (Temp), the set of B lood Pressure (BP) measurements (namely Systemic Artery, Pulmonary A rte ry, Central Venous, Systolic (SB P), Diastolic (DB P) And Me an (MBP) Pr essures, Arterial (ABP )), Maximal Airwa y P ressure (MAP), E xpired Air Volume (EV) , Minute Ventilation (M V) and Transcutaneous Partial Pressures o f Carbon Dio xide (TpCO 2 ) and Oxygen (TpO 2 ). Freq uency o f t he data is restricted by analog to digital sampling capab ility o f t he monitor [ 58]. Storage f requency depends o n d ata lo gging capabilities of bo th t he OEM monitor and the hardware and software mechanism u sed for stor ing stud y data. The synthesis in Tab le II demonstrates that physiologic data are acquired, derived, sam pled and stored at varying frequencies. It is common for AD a lgorithm s to be hard co ded to inp ut particular types of data s treams having speci fic freq uencies. High frequency signals, such as continuous wavefo r ms of ECG, ABP and PP G, are sampled at 100 Hz or more. Data that form lo w frequenc y ti me series, such as HR, SBP , DBP and SpO 2 , are either time -averaged at a rate of once e very second to o nce ever y m inute from high freque ncy si gnals; or RBME- 00025 -2012.R1 3 measured inter mittently every half hour to an h our s uch as temperature and non -invasive B P. D. Data An alysis Analytic aspects o f AD al gorithms are revie wed using the following t hematic taxonomy: Dimensionality, Foc us, Si gnal Quality and Clinical Co ntribution. Theme findi ngs ar e summarised in Table II , in the bottom half of each cell in column three. Dimensio nality represent s the number of variables or data t ypes t hat an al gorithm i s ca pable of analyzing. According to Imhoff et al. [37] , both the cli nical problem and the appro ach to solving it are (a) univariate wh en a single feat ure of a specific data stream is a nalysed; or (b) multivariate when results a re d erived from simultaneous analysis o f multiple variabl es; and in bet ween t hese two approaches lays (c) the univariate clinical pro blem solved using multivariate d ata. A ne w definitio n for multivari ate analysis has e merged in recen t resear ch, a nd as such should be appended to the ab ove a s: (d) an algorithm is multi variate i f it analyses d ifferent metrics der ived from t he same ph ysiologic signal. Exa mples of multivariate type (d) research on the PPG is found in [5 9] and o n multi -lead ECG in [34, 5 0, 60- 63] A uniquely different approac h t o mu ltivariate analysis is found in [64, 65 ], where two differ ent data strea ms were acquir ed from t he same probe making t heir physiologic corr elation easier to exploit. T he Data Analysis theme character izes the focus of eac h algorithm as: (1) strea m; (2) patient; and/o r (3) d isease -centric. A stream-centric algorithm ai ms to indicate, qua ntify or improve sig nal qualit y o f the data strea m for increased reliability. Although t he term p atient -centric has broad implications in health care, it means here that the a lgorithm was trained on patient specific data and is there fore heavily tailored to each sample patient i n the st udy. For e xample, baseline data from a partic ular patient may be req uired to instantiate an algorit hm. Bio Sign [66] is an example o f a rea l- time, auto mated, stream a nd patient-centric s ystem. It produces a single -parameter r epresentation of patient stat us by fusing five d imensions of v ital sign data. A disease -ce ntric methodology focuses on identi fying or predicting a spec ific disease or a cli nically sig nificant outco me. T he brad ycardia detector in [67] and the pro gnostic tool for Late Onset Neonatal Sepsis (LONS) in [68 ] are both exa mples o f stream and disease -centric app roaches. T he clinical co ntribution taxonomy reveals t wo unique con figurations of AD: (i) Standalone AD and (ii) Cou pled A D. As Fig. 1 illustrates, standalone AD tec hniques typically o utput filtered or o riginal physiologic data, annotations and SQI. Standalone techniques are labeled as AD under clinical contribution i n Table II . Coupled AD tec hniques ar e co ded as p art of algorit hms that identify and/or filter artifacts similar to standalone AD techniques with the additional ability for CED or PD. Coupled AD co nfiguration is shown in Fig. 1 and d ocumented under clinical contribution in Table II . Signal Qualit y is a key element in this taxono my. Missing segments, err or, noise and artifacts inevitabl y af fect data quality thus adversel y i mpacting analytic accurac y and reliability [69]. To address this i ssue, Clifford et a l. [70] recommend that an SQI cali brated to provide a kn own er ror rate for a gi ven value of t he SQI be made available for each datum. Nizami et al. [ 71] infer that it suffices for SQIs to be available at a frequency relative to the require ment of another AD, CED or P D applicatio n that co nsume the SQIs. T he lat ter is particularl y relevant when do wn sampled data is required by CED or PDs. It is h ypothesized that the performance o f post processing AD algorithms can improve by consuming streaming SQIs output in parallel for ea ch data strea m by patient monitors [72]. However, use and deliver y of SQI s is no t yet standardised acr oss OEM monitors. For exa mple, in case of a lead disconnectio n the Philips IntelliVue (P hilips, Germany) monitor ou tputs a r andom value above 8 million i n data streams such as the ECG, I RW, HR, RR, SpO 2 and BP. T his also generate s a correspo nding alert type of 'medium prio rity            is qualitative and not quantita tive. T he Delta, Gamma, Vi sta, Kappa and SC6 002 - SC9000 monitor serie s (Dräger Medical Systems, Lubeck, Ger many) output a n SQI value b etween 0 - 100% for the elec troenceph alography (EEG) c hannel. This SQI is calculated u sing sensor imped ance data, artifact information a nd other undisclosed variables. The same monitors also outp ut a fixed la bel 'ARTF' indicati ng artifact on any monitored data stream [73, 7 4]. For example, the monitor classifies Q RS co mplexes onl y at E CG values > 0.20 mV for widths > 70 ms. An artifact condition 'ARTF ' may b e declared when the E CG signal does not m eet these minimum criter ia. Although OEM m onitors may output signal q uality information, there is no logical way to co mpare the SQI s produced by different monitors. Two con founding reaso ns a re: (1) difference in quantificat ion o f SQI; and (2) lack of literature on proprietar y algorithms. T his red uces re search   ability to d etermine how data has b een affected fr om acquisition to log ging. As a result, a post pr ocessing AD algorithm may need to be strictly m atched to i nput d ata sourced from a p articular OEM m onitor as shown i n Fig. 1 . Comparative s tudies led b y Masi mo [75, 76] declar e that its 510 (k) FD A ap proved RADI CAL SET tech nology has the highest quality measure call ed Performance Index ( PI ) as compared to 19 other OEM POs. In these publicati ons, Masimo defines and calculates PI as the percentage of time during which a PO displayed a current SpO 2 value t hat was within 7 % of the simultaneo us co ntrol val ue. However, the Masimo SET technology d oes not auto matically eval uate or log this quantitative SQI. Another q uality measure called Dropout rate (DR) was calculated in [76] , which eq uals the percentage o f measureme nt time d uring which no c urrent SpO 2 values are displa yed. A lthough Masimo SET sh owed equal or worse DR tha n t wo Datex-Oh meda P Os, the rea sons for this data loss are not discussed by Masimo. Independent research groups that compared OEM POs in [55 - 57] neither researched the effect of t he difference in data characteris tics on SQIs nor did they mention if the POs output SQIs. T hrough a recent discourse in [77] on historic developments of the Masimo SET technology, its OEM ha s replied to Va n Der Eij k et a l. [56], claim ing greater accuracy in unstable conditions, such as motio n artifact and lo w perfusion, leading to lo wer false alarm rates. However, [ 77] does not describe any SQI s that can be consumed meaningfully by other A D, CED or PD applications. T his review reco mmends t hat AD algorit hms that produce SQIs, such as PI and DR amongst others, be RBME- 00025 -2012.R1 4 evaluated upon data acquired in [55 - 57] to contribute towards future AD research. It follo ws t hat AD algorith ms desi gned to p ost p rocess OEM monitor data must also consume and d eliver standardised SQIs. In this way another A D algorithm or a CED or PD mechanism can make informed choic es concerning d ata qualit y and validity. Revie w results i n T able II show the increa sing trend i n SQI de velop ment. However, no framework exi sts to unifor mly deliver, compare or combine these SQIs for integratio n with clinical workflows. E. Clinical Eva luation Patient safe ty requires clinical evaluatio n of algorithms prior to rea l-time clin ical implementation. T his theme revie ws clinical evaluation methods b ases on this ta xo nomy: Data Annotation, Mo de and Performance Metric. Res ults are given in Tab le II , in the top half of each cell o f colu mn four. T here are no rules that define gold standard s for clinical eval uation of AD performance. Each s tud y sets its o wn gold standard against which its perfor mance is e valuated. This includ es evaluations of OEM monitors. Annotated physiologic data, where available, ty pically serve as the gold standard for validation studies. Events of interest in the data, s uch as artifacts a nd clinically significant event s, are marked in re al - time or retrospectively. T he onus of perceiving what             d entify the event to the best o f t heir knowledge. Inter -revie wer variability is the sig nificant [78] or subtle [27, 79] difference known to arise when the sa me dataset is annotated b y differ ent reviewers. Retrospecti ve data annotatio n has been s upported by video monitoring i n [27, 31, 36]. Video monitoring is o nly useful w hen the event is visually perce ptible su ch a s s leep movements, certain seizures and routine care. Ho wever, it cannot capture crucial p hysiologic changes s uch as HR deceleration or BP elevation. The advantage of real-ti me annotations i s recording of richer and more accurate cont ent with input fro m sta ff on du ty. Ho wever, this can b e costly a nd requires cooperatio n fro m busy sta ff. Study data is either collected in r eal-time from pa tient monitors o r acquired in an  mode for secondar y analysis from existing datab ases. Review results show that major ity of AD techniques were validated on offline patient data and very few were tested in real-time CCU environments. T able II also documents t he types of performance metrics used in eac h research. I t shows the co mmon trends that will help future resear chers to design and compare differen t al gorithms b y evaluating them using the same metrics. Numeric co mparison of the se perfor mance metrics ca n be found elsewhere in [37 , 4 3, 48, 50 - 52] . Performance metrics need to be interpreted very carefully since statistical signi ficance, or absence thereof, is not al ways representative o f clinical significance, or absence thereof. For example, one missed clinical event ma y not signify a statistical difference in the s ensitivity of o ne OE M monitor over another. Ho wever, the sa me event co uld be very important clinically and c rucial that it not b e missed even once. Theoretica lly, a missed event or a false alar m are caused by artifacts of various t ypes; for example, motion artifact a nd power li ne or optical noise induced in an attached or detached sensor. Studies that collected real-time a nnotated d ata o r vid eo monitored data, such as [27, 31, 36, 80 - 86] among others, ca n utilise the sa me data sets to develop and validate SQIs. Compatible SQIs ca n be used to co mpare perfor mance of different AD al gorithms and OEM monitors. P erformance metrics, for example sensitivity and specificit y in ala r m studies, can be re-eval uated taking into co nsideration the S QI at each alarm instance F. Clinical Imp lementation This theme reviews t he cl inical im pleme ntation status o f AD tec hniques. T heme results are gi ven i n T able II , in the bottom ha lf o f eac h cell in c olumn four. This review re veals that the va st majority o f AD techniques that are published have not been put into cli nical prac tice. This section criticall y reviews implementation o f so me co mmercialized OEM monitors and the very few technique s developed by independent re search groups that made their way into clinical workflows. T he Philips IntelliVue monitoring s yste m (Ro yal P hilips Electronic, Netherlands) feat ures Guardian Ea rly Warnin g Score (EW S) allowing ea ch hospital to choose its own scori ng criteria; Neo natal E vent Review which detect s apnoea, bradycardia and d esaturation; Oxy -cardiorespirog raphy (Oxy- CRG) wit          SpO 2 ; and P rotocolWatch that is clai med to red uce sep sis mortality rates. Presumably Intelliv ue pr eprocesses patient data for ar tifacts prior to CED or P D, however, no valid ation studies or algorithm details o f this s ystem are published. GE Intellirate TM monitor ( Milwaukee, W I, USA) is sues a systole, bradycardia and tachycardia alerts by fusing ECG, ABP and PO d ata. It was eval uated b y GE on a s mall po pulation o f 55 CCU patients in 20 02 [87]. The evaluation was critiqued in [88] for lack of d escription of patient demographics and algorithm specifications. GE has republished the exact same study in 2010. T he Saphire clinical decision s upport s ystem [89] uses Intellirate TM techno logy, b ut do es not e valuate it. Multi-lead ECG arrhythmia detec tion is deplo yed by GE in Datex-Ohmeda Bedside Arrhythmia Monitoring (Mil waukee, WI, USA), MARS Ambulato ry ECG system and MARS Enterprise (Freiburg, Ger many). M ARS uses t he OEM 's E K - Pro Arrhy thmia Detection Algorith m which has been evaluated in over 20 00 monitored hours spanning at least 10 0 patients. Surely, these tech niques fall under the cate g ory of AD coupled with CED and PD. However, literature lacks comparison betwee n different model s marketed b y the same or different OEMs. OEM Covidien-Nellcor (Boulder,CO, US A) has developed RR oxi , a coupled AD and PD technology that derives RR from PO. RR oxi has been validated in real-ti me on 139 healthy s ubjects in [18] . T he OEM is co mmended for this substantial evaluation. Ho wever, healthy s ubjects are not representative of p atient populations which the device is intended to monitor. RR oxi has been validated retrospectively in 12 patients with congesti ve heart failure, by e valuating 20 minutes of data from eac h patient [19]. However, larger studies that i nvestigate p atient po p ulations with several different pathoph ysiologies ar e r equired to conv ince clinicia ns to ad opt another patient monitori ng tec hnology i n their workflows. Fidelit y 1 00 is an FDA 510 (k) approved wireless ECG monitor developed by Signalife (Studio Cit y, C A, USA), which was evaluated in rea l- time i n 54 patients undergoi ng RBME- 00025 -2012.R1 5 percutaneous coronar y intervention [83] . Altho ugh these monitors come w ith different settings applicable for use in different t ypes of CCUs, va lidation studies on p opulation data from all application domains are not found. T here ar e a growing number of online open source ph ysiologic databas es, such as Capnobase [9 0, 91], FDA ECG W arehouse [9 2] , hemodynamic par ameter database [93] , and PhysioNet [94]. It is rec ommended that these databases be used to compare and validate OEM monitors of different makes a nd models. Rest o f this sectio n reviews clin ical impleme ntation of AD research developed b y independent research group s. CIMVA (T herapeutic Monitor ing S ystems Inc., Otta wa, Canada) is a patented multi-organ variability analy tics technology developed by Seel y et al. [95 - 98] . I t is an online tool comprising of multiple coupled AD and CED algorithms with SQIs. Its AD performance is evaluated in [97 ]. CIM VA research can benefit from the recommendations made in the next section regarding common interfaces and formalised SQIs. This will allo w for ne w AD, CED a nd PD al gorithms to be integrated and tested as part of the CIMV A architecture. Otero et al. have i mplemented T RACE, a graphical too l which allows clinician s to edit monitoring rules and criteria in real- time. Coupled AD a nd CED algorithms based on fuzz y set theory input these customized criteria to gen erate patient alarms i n [81, 99], and d etect sleep ap noea in [100]. Given its promising results, evaluation o f TRACE against similar O EM monitors is recommended. T he resear ch conducted in 1999 by Schoenberg et al. [ 54] was integrated as par t o f the commercially a vailable iMDsoft Clinical Information Syste m [101]. Ho wev er, al gorith mic d etails a nd e valuation were ne ver published. Artemis is a real-time d ata anal ytics syste m currently undergoing clinica l evaluatio n in multiple NIC Us around the globe [1 02-104]. As part o f the Artemis framework, Nizami et al. [71 ] present SQI p rocessing to improve the performance of c oupled AD and CED a lgorith ms for LON S. Ongoing Artemis research includes AD [80] ; CED of Apnoea o f Prematurity [5, 15, 105]; as w ell as p ain management [9] . Adoption of the structured app roach to AD recommended in this revie w can enhance t he clinical performance of coupled AD a nd CED in Artemis. T he coupled AD and CED algorit hm for B radycardia b y Portet et al. [ 67] was evaluated on o ffline NICU d ata with the i ntent o f integration with the B abyTalk project. BabyTalk's proof of concept ha s been d escribed in sev eral publication s [1 06- 110] . However, latest research b y Hunter et al. in 2012 [11 1] infers that a long road lies ahead, includin g necessar y clinical tria ls, before BabyTalk co uld be implemented in real -time clinical workflows. Several new AD and CED algorithms can be tested to impro ve outco mes of this proj ect by incorp orating formalised in terfaces as rec ommended in this r eview. The          Charlottesville, VA, US A) t hat scores neo natal Heart Rate Variability (HRV) for predicting LONS is developed using coupled AD and CED al gorithms [6 8]. I t conducts a multivariate type (d) analysis o f the E CG with multi ple coefficients yieldin g a more sensitive res ult [84]. The algorithms i n t his 51 0 (k) FDA app roved device have been extensively described and ev aluated both offline a nd in real - time b y Moor man et al. [8 4, 112- 122] . This p ioneering research has shown pro mising reduction in neonatal morta lity by 2% in a randomized contr ol trial on 3 003 pr eterm babies across nine NI CUs i n the US [11 4]. The drawback o f this tr ial was a 10 % increase in blood work and 5 % more days on antibiotics in t he monitored infants. Ir onically, this co nstitutes the original proble m this resea rch set out to r esolve in [119] . It is recommended that other variabilit y measures, such as those of RR as in [10 ] and PP G as in [123] , be evaluated in             individual scores for each data type; and (b) a co mposite sc ore that e xploits sen sor fusio n f or improved outco mes. B ioSign [66] has b een eval uated retro spectively a nd in r eal -time i n a number of cli nical studie s including randomized control trials in Euro pe and t he US be fore 2006. Ho wever, no later publications could b e located. IV . C ONCLU SIONS This section derives conclusi ons from the the matic revie w. Post processing A D techniques are highly d omain specific. This necessitate s modification for validation and reuse in a different CCU domain. Algorithms may be hard co ded to input OEM sp ecific data t ypes and freq uency. This limits their use with different OEM monitor d ata. T hey may b e validated under certain i nclusion/exclusion cr iteria whic h need to be considered when appl ying th e techniques in other contex ts. Acquisition and sampling frequenc y play an impo rtant role in            may deterio rate to a lif e threateni ng e xtent w ithin s econds. Reusability is d eterred when such implicit limitations are not expressed. T herefore, ado ption of a standardised structured approach for design and repo rting of standalo ne and co upled AD research is r ecommended . Conformity to ge neric input and output i nterfaces will ensure pr esence of all p ertinent information. These inter faces, with co mmon definitions for data type, freq uency, length and SQIs, s hall allo w for matched selection and composition with other AD, CED or PD algorithms. Results from the first three themes are useful in selecting o ne or more AD algorithms that ful fill data requirements of give n CED or PD techniques. Selected AD algorithms can be mixed an d matched to discover o ptimal compositions for varying clinical requirements. RBME- 00025 -2012.R1 6 OEM monitors marketed for use across different CCUs have undi sclosed built-in prep rocessing al gorithms, i nclusive of AD. Moreover, studies valid ating their use in differen t CCU domains and patient population are scarce. The resulting unknown bias imparted in OEM data leads to inevitable variance in analytic results which can effect clinical dec isions. This variability ca n be d ecreased if m onitors o utput comparable standard ised SQIs. As o f yet, SQI s do not conform to any standards and are derived differently in each publication, whether it be SQI s delivered b y OEM p atient monitors or b y AD alg orith ms d eveloped b y indep endent research groups. Inter estingly, none of the revie wed algorithms reported using SQIs provided by OEM monitors. Clinical utility of SQIs ca n b e en hanced b y using formalised definitions s uch tha t SQIs output by different prepro cessing and post processing AD a lgorithms are co mparable as well as compatible. An SQI matched to the sa me set o f de finitions is also pro posed as a requirement at the input of AD, CED or P D algorithms. The objective is to enable the AD, CED or PD algorithm to compare t he inc oming SQIs generated b y o ne o r more AD tec hniques with their required SQI value. The CED or P D algorithm may then ac cept or reject incoming data segments based on fulfillment of the required SQI value. In conclusion, standard ised SQ Is are vital to allo w infor med clinical choices concer ning use and validity o f physiologic data. Results of t he Clinical E valuation the me show t hat majority of AD techniques are validated o n o ffline data a nd very few have been evaluated in real -time CCU enviro nments. Clinical I mplementation theme reveals t hat AD techniques developed by independent resear ch gro ups have rarel y fo und their w ay into clinical implementation. This leads to the inference that a gap exists bet ween research efforts i n AD a nd their utilization in real -time clinical workflows. Whereas real-time cli nical implementation o f AD algorithms is noticeabl y lacking, t here is growing inter est amongst clinicians to use CED and PD for automated clinical decision s upport in CCUs, su ch as in [8, 10 4, 124-135] . Physiologic si gnal q uality a ssessme nt t hrough i ntegration of AD can improve the outcome, reliabilit y and ac curac y of CED and PD research. T he concl usions and recommendations of this re view provide new research dir ection for pro moting integration of AD in real -time clinical workflows. Fig. 1. Patient d ata acquisition by an OEM physiologic monitor with built -in preprocessing inclusive of AD, followed by p ost-processing AD, eith er standalone or coupled with CED a nd/or PD, with output vari ables resulti ng from the e ntire data analy sis. RBME- 00025 -2012.R1 7 T ABLE I: C RITICAL C ARE U NIT (CCU) Theme I: Critical Care Unit ICU PICU NICU OR Other relevant studies Bradley, 2012 [97] Hu, 2012 [136] Li , 2012, 2009, 2008 [20, 21, 59] Sun , 2012 [137] Scalzo, 2012 [60] Borowski , 2011 [138] Siebig , 2010 [27] Schettlinger , 2010 [51] Charbonnier , 2010, 2004 [139, 140] Otero, 2009 [81] Blum, 2009 [141] Aboukhalil , 2008 [88] Otero , 2007 [99] Sieben , 2007 [82] Zong , 2004 [142] Jakob , 2000 [143] Schoenberg , 1999 [54] Ebrahim , 1997 [22] Feldman , 1997 [23] Zhang , 2008 [144] Tsien , 2000, 1997 [85, 145, 146] Ebrahim , 1997 [22] Feldman , 1997 [23] Monasterio , 2012 [147] Hoshik , 2012 [148] Nizami , 2011 [71] Salatian , 2011 [26] Belal, 2011 [149] Blount, 2010 [80] Quinn , 2009 [150] Zhang , 2008 [144] Portet , 2007 [67] Walls-Esquival , 2007 [31] Moorman , 2006 [84] Tsien , 2001, 2000 [145, 151, 152] McIntosh , 2000 [64] Cao , 1996 [65] Karlen , 2012, 2011 [90, 91] Koeny , 2012 [153] Schmid , 2011 [36] Yang , 2010, 2009 [53, 58, 154] Ansermino , 2009 [155] Hoare , 2002 [72] Gostt , 2002 [156] Ebrahim , 1997 [22] Feldman , 1997 [23] Sittig , 1990 [157] Navabi , 1989 [158] Clifford, 2012 [34] Redmond, 2012 [44] Reisner, 2012 [159] Martinez-Tabares , 2012 [160] Silva, 2012 [161] Hayn, 2012 [61] Jekova, 2012 [50] Di Marco , 2012 [62] Johannesen, 2012 [63] Addison , 2012 [18][61] Lázaro, 2012 [17] Otero, 2012 [100] Bsoul , 2011 [162] Xia , 2011 [163] Zaunseder , 2011 [164] Kužílek, 2011 [165] Sukor, 2011, [29] Acharya, 2011 [166] Khandoker, 2011 [167] Nizami, 2010 [168] Nemati, 2010 [169] Alvarez , 2010 [170] Gil , 2008 [171, 172] Chen, 2008 [69] Kostic, 2007 [83] Yu, 2006 [173] Tarassenko, 2006 [66] Fig. 2. Overvie w of the Six Themat ic Taxonomie s CRITICAL CARE UNIT (CCU) ICU PICU NICU OR Other relevant studies Physiologic Data Source Data Monito r Inclusio n/ Exclusion criteria Sample size Harvested Data Streams Data Type Acquisi tion/ /Samplin g or Storage Frequency Data Anal ysis Dimensional ity Focus Signal Qual ity Clinical Contribution Clinical Evaluation Data Annotatio n Mode Performance Metric Clinical Implementation RBME- 00025 -2012.R1 8 T ABLE II : R EVIEWED T HEMES II  VI Author Theme II: Physiologic Data Source Data Monitor/ Inclusion (I)/ Exclusio n (E) criteria/ Sample size Theme III: Harvested Data Data Type/Acquisition (A) / Sampling or Storage (S) Frequency Theme V: Clinical Evaluation Data Annotation / Mode / Performance Metric Theme IV: Data Analysis Dimensionality/ Focus/ Signal Quali ty/ Clinical Contribution Theme VI: Clinical Implementation Bradley [97] Phillips Intellivue MP70 (Philips Healthcare , Andover, Massachusetts)/ I: respiratory and/or cardiac failure, enrollment in the study within 36 hours of ICU admission, expected period on study greater th an 72 hours/ E: chronic atrial fibrillation, transfer from another ICU/ 34 patients ECG, EtCO 2 / A: ECG at 500 Hz; S: 125 Hz Re trospective/Offline/ Perce ntage data loss Univariate/ Stream/ Developed SQI/ AD CIMVA (TMS Inc., Ottawa, Canada) Hu [136] Monitor not specified, data storage system BedMasterEx™/ (i) I: code bl ue adult patients > 18 yr (ii) I: control patients with same: (1) All Patient Refined or Medicare Diagnosis Related Group; (2) age ± 5 years; (3) gender; (4) same CCU / ( ii) E: had code blue; experienced an unplanned ICU transfer/ (i) 223 patients (ii) 1768 patients Alarms/Not applicable Retrospective/ Offline & Simulated real-time / Se, FA Multivariate/ Stream + disease-centric/ No SQI / AD + CED None Li [59] 1 MIMIC II database/ I: asystole & ventricular tachycardia/ 104 patients, total 1055x6s PPG/ A & S: 125 Hz Retrospective/ Offline/ Acc, ROCC Univariate/ Stream + Patient-centric/ Developed SQI/ AD None Sun [137] i - 1 MIMIC II database, ii – BIOPAC PPG 1OOC module, TSD200 PPG reflective transducer, iii - BIOPAC OXY 100C module with TSD123A Sp 0 2 finger transducer / i -87 seconds, ii & iii – 3 healthy volunteers x 20 sec PPG, ECG/ A & S : 125 Hz for MIMIC II datasets;Not specified for BIOPAC data (i ) Retrospective (ii) By real-time observer/ Offline/ RMSE Multivariate/ Stream-centric/ No SQI / AD None Scalzo [60] I: intracranial hypertension/ 108 patients Intracranial Pressure (ICP) & ECG waveforms, ICP alarms/ A: ICP at 240 Hz Retrospective/ Offline/ AUROC, TPR, FPR Multivariate (d)/ Stream-centric/ No SQI/ AD None Borowski [138] Infinity patient monitoring system, Dräger Me dical, (Lubeck, Germany)/ 1245:52:28 hr SBP, MAP, HR, SpO 2 , alarms / S: 250 Hz Retrospective/ Offline/ Se, FARR Univariate/ Stream-centric/ No SQI/ AD None Siebig [27] Infinity patient monitoring system & full-disclosure data logging software eData, (Dräger Medical, Lübeck, Germany)/ 38 patients, total 515 hr At least HR, IABP, SpO 2 , alarms/ S: 1 Hz Retrospective with video/ Offline/ Relevant and FA Univariate/ Stream-centric/ No SQI/ AD None Schettlinger [51] 1 hr of SBP; 30 mins of HR SBP, HR/ A & S: 1 Hz Not specified/ Offline/ Not specified Univariate/ Stream-centric/ No SQI/ AD None Charbonnier [139] Datex-Ensgtrom monitor with a multi-parameter module/ I: various disorders with specific clinical contexts such as mechanical ventilation and cessation of sedative drug administration/ 14 patients, total 50 hr SpO 2 ,SBP, DBP,MBP, HR, MAP, RR,EV, MV, maximal airway flow A: 100 Hz, S: 1 Hz Retrospective/ Offline/ Se, Sp Multivariate/ Stream-centric/ No SQI / AD None RBME- 00025 -2012.R1 9 Author Theme II: Physiologic Data Source Data Monitor/ Inclusion (I)/ Exclusio n (E) criteria/ Sample size Theme III: Harvested Data Data Type/Acquisition (A) / Sampling or Storage (S) Frequency Theme V: Clinical Evaluation Data Annotation / Mode / Performance Metric Theme IV: Data Analysis Dimensionality/ Focus/ Signal Quali ty/ Clinical Contribution Theme VI: Clinical Implementation Li [20, 21] 1 MIMIC II database/ 6000 hr ECG, ABP/ A: 500 Hz; S: 125 Hz Retrospective/ Offline/ True and False Alarms, RMSE Multivariate/ Stream-centric/ Developed SQI / A D + PD None Otero [81] Monitor not specified, data logging software SUTIL/ 78 patients, total 196 hr HR, RR, BP, SpO 2 / Not specified Real-time/ Real-time,TRACE [174] / CDR, FPR Multivariate/ Stream + disease -centric/ No SQI / AD + CED None Blum [141] Solar 9000 monitors and Monitor Capture Server data logg ing software (GE Healthcare, UK)/ 28 days, total 293,049 alarms SBP, MAP, CVP, Chest Impedance (CI) and their alarms/ Not applicable Retrospective/ Offline/ Sp Multivariate/ Stream + patient -centric/ No SQI / AD None Aboukhalil [88] 1 MIMIC II database/ I: critical ECG arrhythmia alarm in the presence of one channel of ECG and an ABP waveforms/ E: 49 patients with active intra- aortic balloon pumps / 5386 alarms from 447 patients, total 41,301 hr ECG, ABP/ A: 500 Hz; S: 125 Hz Retrospective /Offline / True and False Alarm Reduction Rates Multivariate/ Stream + disease -centric/ Specified SQI / AD + CED None Otero [99] OEM not specified / 71 patients, total 175 hr HR, RR, BP, SpO 2 / S: 1 Hz Not specified / Offline, TRACE [174] / CDR, FPR, FNR Multivariate/ Stream-centric/ No SQI / AD None Sieben [82] Not specified RR, SpO 2 , arrhythmia indicator, HR, PR, premature ventricular contraction, SBP, DBP, MBP, temperature, thresholds, alarms/ Not specified Real-time / Offline/ Se, FARR Multivariate/ Stream-centric/ Developed SQI but insufficient details given / AD None Charbonnier [140] OEM Not specified / 18 patients, total 36 hr HR, SBP, DBP, MBP, SAO 2 , MAP / A: 100 Hz, SAO 2 at 0.2 Hz; S: 1 Hz Real-time/ Simulated data/ Number of False, True & Technical Alarms Univariate/ Stream-centric/ No SQI/ AD None Zong [142] 3 MIMIC database / I: at least multi-lead ECG, ABP / 46 patients, total 1890 hr ECG at 500 Hz, ABP at 125 Hz Retrospective / Offline/ TA, FA Multivariate/ Stream-centric/ Calculated SQI / AD None RBME- 00025 -2012.R1 10 Author Theme II: Physiologic Data Source Data Monitor/ Inclusion (I)/ Exclusio n (E) criteria/ Sample size Theme III: Harvested Data Data Type/Acquisition (A) / Sampling or Storage (S) Frequency Theme V: Clinical Evaluation Data Annotation / Mode / Performance Metric Theme IV: Data Analysis Dimensionality/ Focus/ Signal Quali ty/ Clinical Contribution Theme VI: Clinical Implementation Jakob [143] Datex-Ohmeda, (Helsinki, Finland)/ I: systemic and pulmonary artery catheters after coronary artery bypass grafting/ E: clinical signs of heart failure/ 41 patients HR, systemic (SAP) & pulmonary (PAP) artery pressures, SpO 2, central venous pressure (CVP), peripheral & central temp/ A: 10-s median / S: 2- min median values Retrospective/ Offline/ Se, Sp, PPV Univariate/ Stream-centric/ No SQI/ AD None Schoenberg [54] OEM Not specified / 6 patients, total 337 hr HR, SBP, DBP, SpO 2 / Not specified Real-time/ Real-time/ Se, PPV Multivariate/ Stream-centric/ Unknown output score in case of missing data / AD iMDsoft Clinical Information Systems [101] Zhang [144] HP Viridia neonatal component monitoring s ystem (Hewlett Packard)/ 11 patients, total 196 hr HR, PR, RR, SBP, DBP, MBP, SAO 2 , venous O 2 saturation, O 2 perfusion/ A: 1 Hz/ S: 1 Hz and derived 1-min averages Real-time/ Real-time/ Se, Sp, PPV, Acc Multivariate/ Stream + patient -centric/ No SQI / AD None Tsien [145] SpaceLabs monitor (SpaceLabs Medical, Redmond, WA, USA)/ I: monitored data containing all five signals of interest HR, RR, SpO 2 ,Invasive SBP, DBP, MBP / A: 1 sample every 5 s Real-time/ Offline/ AUROC Multivariate/ Stream-centric/ No SQI / AD None Tsien [85] SpaceLabs monitor (SpaceLabs Medical, Redmond, WA, USA)/ E:cardiac PICU patients/ 35118 minutes HR, ECG lead number, RR, SpO 2 , Invasive SBP, DBP, MBP/A & S: 1 sample every 5-6 s Real-time/ Real-time/ TA, FA Not specified/ Stream-centric/ No SQI/ Annotated data for AD None Monasterio [147] 1 MIMIC II database / I: SpO 2 data / 1616 events annotated on 27 patients Two ECG leads, Impedance Pneumogram (IP), PPG each at 125 Hz; HR, SpO 2 at 1 Hz Retrospective/ Offline/ Acc, Sp, Se, PPV, NPV Multivariate/ Stream-centric/ Specified SQI / AD None Hoshik [148] GE Solar 8000M and I and Dash 3000 (GE Healthcare, Milwauke e, USA)/ 1100 patients Three ECG leads at 240 Hz, Chest Impedance (CI) at 60 Hz, PPG,HR,RR, SPO 2 each at 0.5 Hz; alarms, respiratory support, demographics, apnea- bradycardia documentation Retrospective/ Offline/ FPR, FNR Multivariate/ Stream + disease-centric/ Discussed but not specified/ AD + CED None Nizami [71] Not mentioned ECG, PPG Not evaluated Multivariate/Stream + dis ease-centric /SQI utilized/ AD + CED Artemis [104] Salatian [26] OEM Not specified/ 1 data segment BP at 1 Hz Not discussed/ Offline/ Graphical display Univariate/ Stream-centric/ No SQI/ AD None RBME- 00025 -2012.R1 11 Author Theme II: Physiologic Data Source Data Monitor/ Inclusion (I)/ Exclusio n (E) criteria/ Sample size Theme III: Harvested Data Data Type/Acquisition (A) / Sampling or Storage (S) Frequency Theme V: Clinical Evaluation Data Annotation / Mode / Performance Metric Theme IV: Data Analysis Dimensionality/ Focus/ Signal Quali ty/ Clinical Contribution Theme VI: Clinical Implementation Belal [149] Hewlett Packard Merlin M1064a 1176a/ I: neonates < 2 months old/ 54 patients, total 2426 hr HR, RR, SpO 2 / S: 1 Hz Partially retrospective & automated markup/ Offline/ Acc, Sp, Se, ROCC Multivariate/ Stream + disease-centric/ No SQI/ AD + CED None Blount [80] OEM Not specified/ 4 patients Nursing interventions causing artifacts Real-time/ Offline/ Not specified Not specified/ Stream-centric/ No SQI/ AD None Quinn [150] OEM Not specified/ I: 24-29 weeks gestation premature babies in their 1st week of life/ 15 patients x 24 hr Core and Peripheral temp, DBP, SBP, HR, SpO 2 , TpCO 2 , TpO 2 / A & S: 1 Hz Retrospective/ Offline/ AUROC, EER Multivariate/ Stream + disease-centric/ Noticed but not discussed/ AD + CED None Portet [67] OEM Not specified/ I: preterm infants/ 13 patients x 24 hr Core and Peripheral temp, DBP, SBP, HR, SpO 2 , TpCO 2 , TpO 2 , humidity of incubator/ A &S: 1 Hz Retrospective/ Offline / Se, Sp, Acc, ķ Univariate/ Stream + disease-centric/ No SQI/ AD + CED BabyTalk project [111] Walls-Esquival [31] OEM Not specified/ I: preterm infants < 30 wks gestation EEG, ECG, EMG, RR Real-time with video/ Real-time/ Not applicable Multivariate/ Stream-centric/ No SQI / AD Not applicable Moorman [84] Not specified ECG at S: 4 kHz Real-time / Real-time/ AUROC, Positive Predictive Accuracy Multivariate (d)/ Stream + disease-centric/ No SQI/ AD + CED + PD HeRO TM system [114] Tsien [151] OEM Not specified/ 274 hr HR, IBP, TpCO 2 , TpO 2 / A & S: 1 Hz and 1min averages Retrospective/ Offline/ AUROC Multivariate/ Stream-centric/ No SQI / AD None Tsien [152] OEM Not specified/ E: less than four monitored signals / 200 hr HR, BP, TpCO 2 , TpO 2 / S: 1 sample pe r min Retrospective/ Offline/ AUROC Multivariate/ Stream-centric/ No SQI / AD None McIntosh [64] Hewlett Packard 78344A (South Queensferry, U.K.)/ I: pneumothorax/ E: infants with birth asphyxia, persistent pulmonary hypertension, requiring inotropic support/ 42 patients TpCO 2 , TpO 2 / A: 1 Hz & S: 1-min average Retrospective/ Offline/ AUROC,PPV, NPV Multivariate/ Stream + disease – centric/ No SQI /AD + CED None Cao [65] Hewlett Packard 78344 multichannel neonatal monitors/ I: preterm infants/ 10 patients x 10 hr, total 6000 values of PO 2 , PCO 2 TpCO 2 , TpO 2 / A: 1 Hz & S: 1-min average Retrospective/ Offline/ Se, Sp Multivariate/ Stream-centric/ No SQI / AD None RBME- 00025 -2012.R1 12 Author Theme II: Physiologic Data Source Data Monitor/ Inclusion (I)/ Exclusio n (E) criteria/ Sample size Theme III: Harvested Data Data Type/Acquisition (A) / Sampling or Storage (S) Frequency Theme V: Clinical Evaluation Data Annotation / Mode / Performance Metric Theme IV: Data Analysis Dimensionality/ Focus/ Signal Quali ty/ Clinical Contribution Theme VI: Clinical Implementation Karlen [90] (i ) Capnobase database using Nellcor pulse oximeter & S/5 Collect software (Datex-Ohmeda, Finland) (ii) Complex System Laboratory database/ (i) I: general anesthesia/ (i) 124 patients x 120 se c + 42 patients x 480 sec (ii) 2 patients PPG/ (i ) A: 100 Hz & S: 300 Hz Retrospective/ Offline/ Se, PPV Univariate/ Stream-centric/ Developed SQI/ AD None Koeny [153] OEM Not specified/ 17 patients HR, NIBP/ A: 0.33 Hz; S: 0.0167 Hz Real-time/ Offline/ Graphical display Univariate/ Stream-centric/ No SQI/ AD None Karlen [91] Capnobase database using S/5 Collect software (Datex-Ohmeda, Finland)/ I: general anesthesia/ 42 patients ECG, PPG, RR/ A: ECG at 300 Hz, PPG at 100 Hz; RR at 25 Hz; S: 300 Hz Retrospective/ Offline/ Error, Power, Robustness Multivariate/ Stream-centric/ Discussed but not specified/ AD None Schmid [36] Kappa XLT monitor (Dräger), Zeus anesthesia workstation (Dräge r), Nortis MedLink, (Lubeck, Germany), Erasmus MC eData TapeRec, (Rotterdam, The Netherlands)/ I: anesthesia for elective cardiac surge ry (aortocoronary bypass grafting and valve surgery)/ 25 patients ECG, PPG, SBP, MAP, DBP, CVP, LAP, SpO 2 , Temp, MV, RR, PAW, CO 2 exp , CO 2 insp , Isoflurane Insp , Isoflurane exp , alarms/Not specified Retrospective with video/ Offline/ Alarm Validity Univariate/ Stream-centric/ No SQI/ AD None Yang [53] GE S/5 Monitor/ 40 patients EtCO 2 , MAP, MV exp, NIBP mean/ A & S: 1 s ample every 5 sec Retrospective/ Offline/ TPR, FPR Univariate/ Stream-centric/ No SQI/ AD None Yang [58] OEM Not specified/ 10 patients NIBP mean / A & S: 1 sample every 5 sec Retrospective/ Offline/ TPR, FPR Univariate/ Stream-centric/ No SQI/ AD None Ansermino [155] GE S/5 Monitor/ 47 surgeries x 1 hr (19 children, 28 adults) HR, NIBP mean, SpO 2 , EtCO 2 , MV exp, RR/ Not specified Real-time/ Offline/ Se, PPV, NPV, TPR, FPR Univariate/ Stream-centric/ No SQI/ AD None Yang [154] GE S/5 Monitor/ E: fibrillation and other instances of abnormal heart rhythm/ 2 patients HR, PR, IBP/ A: 1 sample every 5 sec Retrospective/ Simulated data & [155] / RMSE Multivariate/ Stream-centric/ No SQI / AD None Hoare [72] Local hospital database, OEM Not spe cified / 245 cases HR/ A & S: 1 sample every 30 sec Retrospective/ Offline/ AUROC, FPR, PPV Univariate/ Stream-centric/ No SQI/ AD None Gostt [156] Datex AS/3 Anaesthesia Monitor, Nellcor N- 200/ 9 paediatric and 11 adult patients HR, PR/ A: 1 sample every 5 sec, but not always due to data capture issues Retrospective/ Real-time/ Acc, Se, Sp Multivariate analysis for univariate problem/ Stream + disease-centric/ No SQI/ AD + CED None RBME- 00025 -2012.R1 13 Author Theme II: Physiologic Data Source Data Monitor/ Inclusion (I)/ Exclusio n (E) criteria/ Sample size Theme III: Harvested Data Data Type/Acquisition (A) / Sampling or Storage (S) Frequency Theme V: Clinical Evaluation Data Annotation / Mode / Performance Metric Theme IV: Data Analysis Dimensionality/ Focus/ Signal Quali ty/ Clinical Contribution Theme VI: Clinical Implementation Sittig [157] OEM Not specified / I: during open heart surgery/ 1 patien t HR, HR from ABP, Pulmonary artery mean pressure, MAP/ A: 1 sample every 2 min S: 1 sample repeated every sec Not specified/ Real-time/ Graphical evaluation Univariate/ Stream-centric/ No SQI/ AD None Navabi [158] 5 kinds of Datascope monitors, Nellcor PO, Critikon VRP respiratory monitor/ 21 surgical cases HR, EtCO 2 , Inspired O 2 , NIBP, SAO 2 , HR, Inspired & expired lung volumes, RR, MAP, MV/ A: EtCO 2 , Inspired O 2 , NIBP, HR, SAO 2 : 0.2 Hz, Inspired & expired lung volumes, RR, MAP, MV: 0.0167 Hz Not specified/ Real-time/ CDR, FARR Multivariate/ Stream-centric/ No SQI / AD None Ebrahim [22] Feldman [23] SpaceLabs Medical Gateway, SpaceLabs Medical PC2/ 12 OR, 60 adult ICU, 13 PICU patients, each ICU record is 4 h ours long HR, BP, PR, SpO 2 / A & S: HR, PR reading every 3- 5 s Real-time/ Real-time in [23] / False and missed alarms Multivariate analysis for univariate problem/ Stream centric/ No SQI/ AD+PD None Clifford [34] 2 Sana Project database / 30, 000 x 10 sec Twelve lead ECG/ A & S: 500 Hz Retrospective/ Offline/ Acc, ROCC Multivariate type (d)/ Stream-centric/ Developed SQI / AD None Redmond [44] TeleMedCareHealth Monitor (TeleMed Care Pty Ltd Sydney, Australia) / I: home dwelling patients with chronic obstructive pulmonary disease and/or congestive heart failure/ 288 patients, total 300 recordings Single Lead I-ECG/ S: 500 Hz Retrospective/ Offline/ Se, Sp, Acc, ķ Univariate/ Stream-centric/ SQI can be developed/ AD None Reisner [159] Propaq 206EL monitors (Protocol Systems, Beaverton, Ore)/ I: prehospital trauma adult patients/ 671 patients ECG, RR, HR, SBP, DBP/ A & S: ECG at 182 Hz; HR, RR at 1 Hz Retrospective/ Offline/ AUROC Multivariate / Stream + disease-centric/ No SQI/ AD + CED None Martinez- Tabares [160] (i) INNOVATEC S.L. (Spain), CELBIT LTDA. ELECTRODOCTOR (Colombia), Lionheart 1 (BIO-TEK) simulator, QRS-Card (Pulse Biomedical Inc) (ii) 2 Sana Project database / ( ii) 1000 x 12 sec Twelve lead ECG/ A & S: 500 Hz Retrospective/ Offline/ Se, Sp, Acc, ROCC Multivariate / Stream-centric/ Developed SQI / AD None Silva [161] 1 MIMIC II database / 1361 epochs Multi-lead ECG, PPG, ABP, Resp, CVP/ S: 125 Hz Retrospective/ Offline/ AUROC Multivariate/ Stream-centric/ Developed SQI / A D None RBME- 00025 -2012.R1 14 Author Theme II: Physiologic Data Source Data Monitor/ Inclusion (I)/ Exclusio n (E) criteria/ Sample size Theme III: Harvested Data Data Type/Acquisition (A) / Sampling or Storage (S) Frequency Theme V: Clinical Evaluation Data Annotation / Mode / Performance Metric Theme IV: Data Analysis Dimensionality/ Focus/ Signal Quali ty/ Clinical Contribution Theme VI: Clinical Implementation Hayn [61] 2 Sana Project database / 20 00 x 10 sec Twelve lead ECG/ A & S: 500 Hz Retrospective/ Offline/ Se, Sp, Acc Multivariate type (d)/ Stream-centric/ Developed SQI / AD None Jekova [50] 2 Sana Project database / 15 00 x 10 sec Twelve lead ECG/ A & S: 500 Hz Retrospective/ Offline/ Se, Sp, ROCC Multivariate type (d)/ Stream-centric/ Developed SQI / AD None Di Marco [62] 2 Sana Project database / 1498 x 10 sec Twelve lead ECG/ A & S: 500 Hz Retrospective/ Offline/ Acc, ROCC Multivariate type (d)/ Stream-centric/ Developed SQI / AD None Johannesen [63] 2 Sana Project database / 1500 x 10 sec Twelve lead ECG/ A & S: 500 Hz Retrospective/ Offline/ Acc, ROC curves Multivariate type (d)/ Stream-centric/ Developed SQI / AD None Addison [18] Nell-1 oximeter module, (Covidien-Nellcor, Boulder, CO, USA) with a Nellcor Max-A disposable probe, Datex-Ohmeda CardioCap/S5/ 139 healthy volunteers x 8 min PPG, EtCO 2 / A: PPG at 75.7 Hz Real-time/ Real-time/ Root Mean Square Difference (RMSD) Univariate/ Stream-centric/ No SQI/ AD + PD RR OXI (Covidien-Nellcor,Boulder,CO, USA) Lázaro [17] Biopac OXY100C,ECG100C, RSP100C sens or, TSD201 transducer & Finometer / 17 subjects x 9 min PPG at 250 Hz, ECG leads I, III & precordials at 1000 Hz, respiratory signal r(n) at 125 Hz, BP at 250 Hz Not specified/ Offline/ Inter-subject mean & standard deviation Multivariate/ Stream-centric/ No SQI / AD + PD None Otero [100] Polysomnographic device, ( Nicolet Biomedical Inc.)/ I: sleep study/ 10 patients, total 59 hr 10 min Respiratory airflow, SpO 2 / A: 68.25 Hz; S: 4 Hz Retrospective/ Offline, TRACE [174] / CDR, FPR Multivariate / Stream + disease-centric/ No SQI/ AD + CED None Bsoul [162] PhysioNet Apnea-ECG database/ 35 subjects Polysomnography and ECG/ A: ECG at 100 Hz, S: ECG at 250 Hz Retrospective/ Offline/ F-measure, Se, Acc, Multivariate / Stream + disease-centric/ No SQI/ AD + CED + PD None Xia [163] 2 Sana Project database / 1000 x 10 sec Twelve lead ECG/ S: 500 Hz Retrospective/ Offline/ Acc, Se, Sp Multivariate type (d)/ Stream-centric/ Discussed SQI / AD None Zaunseder [164] 2 Sana Project database / 1000 x 10 sec Twelve lead ECG/ S: 500 Hz Retrospective/ Offline/ Acc Multivariate type (d)/ Stream-centric/ Developed SQI / AD None RBME- 00025 -2012.R1 15 Author Theme II: Physiologic Data Source Data Monitor/ Inclusion (I)/ Exclusio n (E) criteria/ Sample size Theme III: Harvested Data Data Type/Acquisition (A) / Sampling or Storage (S) Frequency Theme V: Clinical Evaluation Data Annotation / Mode / Performance Metric Theme IV: Data Analysis Dimensionality/ Focus/ Signal Quali ty/ Clinical Contribution Theme VI: Clinical Implementation Kužílek [165] 2 Sana Project database / Not clear Twelve lead ECG/ S: 500 Hz Retrospective/ Offline/ Not clear Multivariate type (d)/ Stream-centric/ No SQI / AD None Sukor [29] MLT1020FC Reflection mode infrared finger probe, Differential bio - amplifier ST4400, PowerLab data acquisition system (ADIns truments, Sydney, Australia) / 13 healthy subjects PPG, ECG/ A: 1 kHz Retrospective & Real-time/ Offline/ Se, Sp, Acc, ķ Multivariate validation of a univariate classifier/ Stream-centric/ SQI can be developed/AD + PD None Acharya [166] Sleep staging signals Grass amplifiers (Astro-Med Inc., USA), other OEM not specified / 25 subjects with suspected disease , 14 normal subjects ECG/ A & S: 256 Hz Real-time PSG annotations by a clinician/ Offline/ Acc, Se, Sp, PPV using confusion matrix Univariate/ Stream + disease-centric/ No SQI/ AD + CED None Khandoker [167] OEM Not specified / I: sleep apnoea & E: cardiac history / 29 patients EEG, electrooculograms, ECG,leg movements, body positions, thoracic & abdominal wall expansion, oronasal airflow, SpO 2 / S: 250 Hz for ECG, S: 32 Hz for thoracic and abdominal wall expansion Polysomnography annotations/ Offline/ Acc, Se, Sp Multivariate validation of a univariate classifier/ Stream + disease-centric/ No SQI/ AD + CED None Nizami [168] PhysioNet databases: nsrdb; MIT-BIH: nsr2db, mitdb, svdb; BIDMC chfdb, chf2db/ I: various records with normal sinus rh ythm, arrhythmia o r congestive heart failure/100 patient records RR Interval / Not applicable Retrospective/ Offline/ Acc Univariate/ Stream + disease-centric/ No SQI/ AD + CED None Nemati [169] Peripheral Arterial Tonometry (PAT) measurement (Itamar Med ical, Isreal), other OEMs not specified / I: apnoea-hypopnoea index (AHI) of 0 – 69.3 events/hour 30 patients x 6-8 hr ECG, 4 channels of respiratory rate (from chest & abdomen plethysmograph, nasal & oral thermistor, nasal pressure), PAT / A & S: 500 Hz, except PAT at 100 Hz Polysomnography annotations/ Offline/ Signal- to -Noise Ratio Multivariate/ Stream-centric/ Developed and implemented SQI/ AD None Alvarez [170] Polysomnograph (Alice 5, Respironics, Phil ips Healthcare, The Netherlands), Nonin PureSAT PO (Nonin Medical, Plymouth, MN, USA)/ I: daytime hypersomnolence, loudsnoring , nocturnal choking and awakening, apneic events/ E: other sleep disorders e.g ., insomnia, parasomnia, narcolepsy/ 148 patients x 8 hr SaO 2 / S: 1 Hz Real-time Polysomnography annotations by a clinician/ Offline/ Acc, Se, Sp, ROCC Univariate/ Stream + disease-centric/ No SQI/ AD + CED None RBME- 00025 -2012.R1 16 Author Theme II: Physiologic Data Source Data Monitor/ Inclusion (I)/ Exclusio n (E) criteria/ Sample size Theme III: Harvested Data Data Type/Acquisition (A) / Sampling or Storage (S) Frequency Theme V: Clinical Evaluation Data Annotation / Mode / Performance Metric Theme IV: Data Analysis Dimensionality/ Focus/ Signal Quali ty/ Clinical Contribution Theme VI: Clinical Implementation Chen [69] Propaq EncoreR 206EL monitor (ProtocolR Systems Inc.)/ I: hemorrhaged patients receiving blood with documen ted injury/ E: insufficient documentation of injury / 823 patients at least one of HR, RR, DBP, SBP, SaO 2 A & S: 1 Hz, except BP taken intermittently Retrospective/ Offline/ AUROC Multivariate / Stream + disease-centric/ Developed SQI / AD + CED None Kostic [83] ECG recorders: Signalife Fidelity 100, NorthEast DR180+, Mortara ELI 200, HP Page Writer 1700A & GE MAC 5000/ I: percutaneous coronary intervention (PCI)/ 54 patients ECG/ S: 720 Hz Real-time/ Real-time/ Qualitative, graphical assessment Univariate/ Stream + disease-centric/ No SQI/ AD + CED None Yu [173] Propaq EncoreR 206EL monitor (ProtocolR Systems Inc.)/ I: trauma patients during helicopter transport to a hospital/ 726 pati ents x 25 mins ECG waveform at 182 Hz, PPG waveform at 91 Hz, HR, PR at 1 Hz Retrospective/ Offline/ AUROC Multivariate / Stream + disease-centric/ Developed SQI / AD + CED None Gil [171, 172] (i) ECG Apnea Data Base from Physionet; (ii) Digital polyg raph EGP800 (Bitmed), COSMO EtCO 2 /SpO 2 Monitor(Novametrix, Medical Systems)/ (i) I: children suspected of having Obstructive sleep apnea syndrome (OSAS)/ (ii) E: lack of manual annotations, unacceptable resp iratory flow signal quality, doubtful clinical diagnosis/ (i) 70 adults (ii) 26 children Polysomnography data: a chin electromyogram, 6 EEG & 2 electro-oculogram channels, ECG, air flow, respiratory plethysmography, PPG, SaO 2 , CO 2 / S: all signals at 100 Hz Retrospective/ Offline/ Se, Sp, PPV Multivariate / Stream + disease-centric/ No SQI / AD + CED None Tarassenko [66] OEM not specified / I: patients monitored for at least 24 hr after a myocardial infarct and again for a few hours 5 days later; patien ts w ith severe heart failure; patients with acute respiratory proble ms; elderly patients with hip fracture/ 150 patients x 24 hr HR, RR,SpO 2 ,temp, BP/ A & S: 1 sample every min Retrospective & Real-time/ Real time, BioSign system/ Percentage of True alarms Multivariate/Stream + patient-centric/ No SQI/ AD None 1 PhysioNet MIMIC II database recorded using Philips Intellivue M P -70 Medical Systems; 2 PhysioNet database: Sana Project, OEM not specified ; 3  using  bedside monitor. Abbreviations and acronyms : Accuracy (Acc); Sensitivity (Se); Specificity (Sp); Receive r Operating Characteristic Curve (ROCC); Area Under the ROC Curve (AUROC); Positive Predictive Value (PPV); Nega tive Predictive Value (NPV); Correct Det ection Rate (CDR); False Positive Rate (FPR); False Negative Rate (FNR); True Alarm Rate (TA); False Alarm Rate (FA); Root Mean Square Error (RMSE); False A larm Reduction Rate(FARR); Equal Er   Electrooculograms (EOG). RBME- 00025 -2012.R1 17 R EFERENCES [1] F. Musial, "W hy should w e integrate biomarke rs into com plex trials? " Forsch. Komplem entarmed., vol. 19, pp. 232-233, 2012. [2] A. Bravi, A . Longtin and A . J. E. Se ely, "Review and classification of variability analy sis techniques w ith clinical ap plications," BioMedical Engineering On line, vol. 10, 2011. [3] J. Fackle r and M. Spaeder, "W hy doe sn't healthcare embra ce simulatio n and model ing? What woul d it take?" Proceedings - Winter Simulation Conference, pp. 1 137-1142, 2011. [4] J. Cirell i, B. Gray don, C. McGregor and A . James, "Analy sis of continuous oxy gen saturation da ta for accurate represe ntation of retinal exposure to o xygen in the prete rm infant," in Inform ation Technolo gy and Communications i n Health, 2013 (in press) . [5] A. Thomm andram, J. E. Pugh , J. M. Eklun d, C. McGr egor and A. G. James, "Classify ing Neonatal Spel ls Using Real-T ime Temporal Analy sis of Physiol ogical Data Streams: A lgorithm Dev elopment " IEEE EMBS Speci al Topic Conference o n Point- of -Care Healthc are Technologies, 2013 (accepted paper). [6] A. T. Cruz, E . A. Wil liams, J. M. G raf, A. M. Perry , D. E. Harbin, E. R. Wuestner and B . Patel, "T est characteristics of an automate d age- and temperature-adjuste d tachy cardia alert in pediatric sep tic shock," Pediatr. Emerg. Care, vol. 28, pp. 889-894, 2012 . [7] T. Smith, D. De n Hartog, T. Moerman, P. P atka, E. M. M. V an Lie shout and N. W. L . Schep, "Accura cy of an e xpanded early warning score for patients in gener al and trauma s urgery wards," Br. J. Sur g., vol. 99, pp. 192- 197, 2012. [8] A. Jalali, P. Ghorbanian, A . Ghaffari an d C. Nataraj, "A novel technique for identify ing patients with I CU needs using hemodynami c features," Advances in Fuzz y Systems, 2012. [9] N. Bressan, C . McGregor , M. Blount, M. Ebling, D. Sow and A . A. Jame s, "Identificatio n of Noxious Even ts For Newbor n Infants With a N eural Network," 4th Con gress of the Eur opean Academy of Paediatric S ocieties, 2012. [10] C. McG regor, C. Catley and A. James, "V ariability analy sis with analytics applied to physiol ogical data streams fr om the neonatal intensive care unit," Proceedi ngs - IEEE Sym posium on Computer-Bas ed Medical Systems, 2012 . [11] C. McG regor, C. Catley and A. James, "A process mining drive n framewo rk for clinical guidel ine improv ement in critical care ," in Learning from Medical D ata Streams LEM EDS'11,13th Con ference on Arti ficial Intelligence in Medicine, Slove nia, 2012 . [12] K. A. Batc helder, P. D. Ma nnheimer, R. S. Mecca and J. M. Ojile, "Pulse oximetry saturatio n patterns de tect repetitive reductions in airflow ," J. Clin. Monit. Comput., vol . 25, pp. 411-418, 2011. [13] A. M. Saw yer, E. N. Deal, A . J. L abelle, C. Witt, S. W. T hiel, K. He ard, R. M. Reichle y, S. T. Micek an d M. H. Ko llef, "Impleme ntation of a real-time computerize d sepsis alert in nonintensive care unit patients," Crit. Care Me d., vol. 39, pp. 469-4 73, 2011. [14] P. Ordóñez , T. Armstro ng, T. Oates and J . Fackler, "U sing modified multivariate bag- of -w ords models to cl assify physio logical data," Proceedings - IEEE Internat ional Conference on Data Mining , ICDM, pp. 534-539, 2011. [15] C. Catley , K. Smith, C. McG regor, A . James and J. M. Ekl und, "A Framewo rk for Multidimensional Rea l-Time Data Analysis: A Case Study for the Detection of Apnoea of Pre maturity ," International Journal of Computational Models and Alg orithms in Medicin e, vol. 2, p p. 16-37, 2011 . [16] M. Al tuve, G. Carrault, A . Beuchee , P. Pladys and A. I. He rnández, "On- line apnea-brady cardia detection using hidden se mi-Markov models," Proceedings of the Annual Int ernational Conf erence of the IEEE Engineer ing in Medicine an d Biology Societ y, EMBS, pp. 4374-4377, 2011 . [17] J. L ázaro, E. Gil, R. Bailón, A . Mincholé and P. Laguna, "De riving respiration from photoplethy smographic pulse width," Medical and Biol ogical Engineering an d Computing, pp. 1- 10, 2012. [18] P. S. Ad dison, J. N. Watso n, M. L . Mestek and R. S. Mecca, "Devel oping an algorithm fo r pulse oximetry derived respiratory rate (RR oxi): A healthy volunteer study," J. Clin. Monit. Com put., vol. 26, p p. 45-51, 2012. [19] M. Mestek, P. Addison, A . Neitenbach, S. Bergese and S. K elley , "Accuracy of Continuous No ninvasive Respira tory Rate De rived From Pulse Oximetry in Conge stive Heart F ailure Patients " C HEST, 20 12. [20] Q. L i, R. Mark and G. Cliffo rd, "A rtificial arterial blo od pressure artif act models and an ev aluation of a ro bust bloo d pressure and heart r ate estimator," BioMedical Engi neering OnLine, vo l. 8, pp. 13, 20 09. [21] Q. L i, R. G. Mark and G . D. Cliffor d, "Robust heart rate estimation from multiple asy nchronous noisy sources using sig nal quality indice s and a Kalman filter ," Physiol. Meas., vol. 29, pp. 15-32, 2 008. [22] M. H. Ebrahim, J . M. Fel dman and I. Bar-K ana, "A robust sensor f usion method for he art rate estimatio n," J. Clin. Monit., vol . 13, pp. 385-393, 1997. [23] J. M. Fe ldman, M. H. Ebrah im and I . Bar -Kana, "Rob ust sensor fus ion improves heart ra te estimation: Cli nical ev aluation," J. Clin. Monit., vol . 13, pp. 379-384, 1997. [24] D. M. Mirvis , A. S. Berso n, A. L. G oldberger, L . S. Green, J. J. Hege r, T. Hinohara, J. I nsel, M. W. K rucoff, A. Moncr ief, R. H. Selv ester and G. S. Wagner , "Instrumentation and pr actice standar ds for ele ctrocardiographic monitoring in spec ial care units. A report for heal th professio nals by a task force of the council on clinical cardio logy, A merican Health A ssociation," Circulation, vol. 79, pp. 464-471, 198 9. [25] K. E. Ray mer, J. Berg ström and J. M. Ny ce, "Anaesthesia mo nitor alarms: a theo ry-driven approach," Er gonomics, vol. 55, pp. 1487-1501, 2 012. [26] A. Salatian and L. Oborkhale, "F iltering of I CU monitor data to re duce false alarms and e nhance clinical decision support ," International Journal of Bio-Science and Bio-Technology, vol . 3, pp. 49-55, 2011. [27] S. Siebig, S. Kuhls, M. I mhoff, J. L anggartner, M. Reng, J. S chölmerich, U. Gather and C. E. Wre de, "Collection of annot ated data in a clinical validation study for alarm al gorithms in intens ive care-a methodolo gic framewo rk," J. Crit. Care, vol. 25, pp. 128-135, 2010. [28] G. Takla, J. H . Petre, D. J. D oyle, M. Hor ibe and B. Gopakumaran, "The Problem o f Artifacts in Patient Monitor Data D uring Surgery : A Clinical and Methodolo gical Review ," Anesth. Analg., vol. 103, pp. 1196-1204, 2006. [29] J. A. Sukor , S. J. Redmond and N. H. L ovell, "Signal quali ty measures for pulse oxime try through wav eform morpholo gy analy sis," Ph ysiol. Meas ., vol. 32, pp. 369-3 84, 2011- 03 -01, 2011. [30] L. A . Lynn and J. P. Curry , "Patte rns of unexpected in-hospital deat hs: A root cause anal ysis," Patient Safety in Surgery, vo l. 5, 2011. [31] E. Wall s-Esquivel, M. F. Ve cchierini, C. Hé berlé and F. W allois, "Electroe ncephalography (EEG ) recording tec hniques and ar tefact detection in early pre mature babies," Neurophys iologie Cliniq ue/Clinical Neurophysiology , vol. 37, pp. 299-30 9, 2007. [32] M.      Hermosillo , "Common ele ctrocardiographic artifa cts mimicking arrhy thmias in ambulatory monitoring," Am. Hear t J., vol. 144, pp. 187-197, 2002. [33] C. Chase a nd W. J. Brady , "Artifactual el ectrocardiographic change mimicking clinic al abnormality on the ECG," Am. J. Em erg. Med., vol . 18, pp. 312-316, 2000. [34] G. D. Cliff ord, J. Behar, Q . Li and I . Rezek, "Signal q uality indices an d data fusion for determining cli nical acce ptability of e lectrocardi ograms," Physiol. Meas., vol . 33, pp. 1419, 2012. [35] A. Ko nkani, B. Oakley and T. J. Ba uld, "Reducing hospital noise: A review of medical device alarm m anage ment," Biomedical Instrume ntation and Technology , vol. 46, pp. 478-487, 2 012. [36] F. Schmid, M. S. Goe pfert, D. Kuhnt, V. Eich horn, S. Diedri chs, H. Reichenspurner, A . E. Go etz and D. A . Reuter, "The wol f is crying in the operating roo m: Patient monitor a nd anesthesia w orkstation alarmi ng patterns during cardiac s urgery ," Ane sth. Anal g., vol. 112, pp. 78-83 , 2011. [37] M. Imhof f, S. Kuhls, U. G ather and R. F ried, "Smart alarms fr om medical devices i n the OR an d ICU," Best Practice & Research Clinical Anaesthesiolo gy, vol. 23, pp. 39-50, 2 009. [38] S. Siebig, W . Sieben, F. K ollmann, M. I mhof, T. Bruennler, F. Rockmann, U. G ather and C. E. W rede, "Use rs' opinions on intens ive care unit alarms - A surve y of German intensive care units," Anaesth. Inte nsive Care, vol. 37, pp. 1 12-116, 2009. [39] J. M. Sole t and P. R. Barac h, "Managing alarm fatigue in cardiac care, " Prog. Pediatr. Cardiol., vol. 33, pp. 85-90, 2012. [40] M. Cvach, "Mo nitor alarm fa tigue: An integrative re view," Biomedical Instrumentation and Technol ogy, vol. 46, pp. 268-277, 2 012. [41] M. H. Hoope r, L. We avind, A. P. W heeler, J. B. Mar tin, S. S. G owda, M. W. Semle r, R. M. Hayes, D. W . Albe rt, N. B. Deane, H. Ni an, J. L . Mathe, A. Nadas, J. Szti panovits, A. Mille r, G. R. Ber nard and T. W . Rice, "Randomiz ed trial of automate d, electronic moni toring to facili tate early detection of sepsi s in the intensive care unit," Crit. Care Med., vol. 40, pp. 2096-2101, 2012. [42] H. Xia, G . A. Garcia an d X. Zhao, "A utomatic detectio n of ECG electrode mispla cement: a tale of two alg orithms," Physiol. Meas., vo l. 33, pp. 1549, 2012. [43] G. D. Cliff ord and G. B. Moo dy, "Sig nal quality in car diorespiratory monitoring," Physi ol. Meas., vol. 33, 2012. [44] S. J. Redmo nd, Y. Xie, D. Chang, J. Basilak is and N. H. L ovell, "Electrocardiogr am signal quality me asures for unsupe rvised tele health environments," Physiol. Meas., vol . 33, pp. 1517-1533, 20 12. RBME- 00025 -2012.R1 18 [45] S. J. Redmo nd, Q. Y. L ee, Y. Xie and N. H. L ovell, "Applications of supervised learni ng to biolo gical signals: ECG signal quality and sy stemic vascular resistance ," Proceedings of the Annual I nternational Confer ence of the IEEE Engi neering in Medici ne and Biol ogy Society, EMB S, pp. 57-60, 2012. [46] I. Silv a, G. B. Moody and L. Celi, "I mproving the quality of ECGs collected usi ng mobile phones: T he PhysioNet/Computing i n Cardiology Challenge 2011," Computers in C ardiology, vol . 38, pp. 273-276, 2011. [47] M. Borow ski, M. Görg es, R. Fried, O. Suc h, C. Wrede and M. Imhoff, "Medical device alar ms," Biomed. Te ch., vol. 56, pp. 7 3-83, 2011. [48] K. Schettlinge r, R. Fried and U. G ather, "Robust filters fo r intensive care monitoring: Bey ond the running me dian," Biomed. Tec h., vol. 51, pp. 49-56 , 2006. [49] M. Chambri n, "Al arms in the intensive care unit: How can the number of false alarms be reduced?" Critical Ca re, vol. 5, pp. 184-18 8, 2001. [50] I. Je kova, V. Krastev a, I. Christov and R. Ab, "Threshol d -based syste m for noise detect ion in multile ad ECG reco rdings," Physiol. Meas., vol . 33, pp. 1463, 2012. [51] K. Schettli nger, R. Fried a nd U. G ather, "Real-time signal processi ng by adaptive repe ated median filters," I nt J Adapt Control S ignal Process, vol. 24, pp. 346-362, May 2010. [52] M. Imhof f and S. Kuhls, "A larm algo rithms in critical mon itoring," Anesth. Analg. , vol. 102, pp. 1525-15 37, 2006. [53] P. Yang, G . Dumont and J. M. Ansermino, "A Cusum-Based Multile vel Alerting Metho d for Physiol ogical Monitoring," Informa tion Technology in Biomedicine, I EEE Transactions on, vol. 14, pp. 1046-105 2, 2010. [54] R. Schoen berg, D. Z. Sands a nd C. Safran , "Making ICU alarms meaningful: a com parison of tr aditional vs. tren d-based algorithms." Proceedings An nual Symposium AMIA, pp. 379-383, 1999. [55] E. D. Johnsto n, B. Boy le, E. Juszczak, A . King, P. Brockle hurst and B. J. Stenson, "Oxy gen targeting in pre term infants using the Masimo SET R adical pulse oximete r," Archives of Diseas e in Childhood: Fe tal and Neonat al Edition, vol. 96, pp. F429-F433, 201 1. [56] A. C. Van De r Eijk, S. Ho rsch, P. H. C. Eil ers, J. Dankel man and B. J. Smit, ""New -generation" pulse ox imeters in extre mely low-birth-we ight infants: How do they per form in clinical pract ice?" Journal of Perinatal and Neonatal Nursin g, vol. 26, pp. 172- 180, 2012. [57] D. L i, V. Jeyaprakash, S. F oreman and A . M. Groves, "L etters: Comparing oxy gen targeting in pre term infants betw een the Masimo and Philips pulse ox imeters," Arch Dis C hild Fetal Neo natal Ed, vol. 97:4, pp. F311-F312, 2012. [58] P. Yang, G . A. Dumont an d J. M. A nsermino, "Online pattern recog nition based on a gene ralized hidde n Markov mode l for intraoperative vital sign monitoring," Int J A dapt Control Si gnal Process, vol . 24, pp. 363-381, May 2010. [59] Q. L i and G. D. Clifford, "D ynamic time w arping and mac hine learni ng for signal quality assessme nt of pulsatile signals," Physiol. Me as., vol. 33, pp. 1491, 2012. [60] F. Scalz o, D. Liebeskind an d X. HU, "Re ducing False Intracranial Pressure A larms using Morphol ogical Wave form Features," Biomedical Engineering, IEE E Transactio ns on, vol. PP, pp. 1-1, 201 2. [61] D. Hay n, B. Jammerbund and G. Schreier, "QRS detection based ECG quality assessme nt," Physiol. Me as., vol. 33, pp. 1449-1461, 2012. [62] L. Y. Di Mar co, W. Duan, M. Bojarnej ad, M. Zheng, S. K ing, A. Murr ay and P. L angley, "Evaluation of a n algorithm b ased on single-condition decision rules fo r binary classificatio n of 12-lead am bulatory ECG recording quality," Physiol. M eas., vol. 33, pp. 1435, 2012. [63] L. G . Lars Johannesen a nd, "Automatic E CG quality scoring methodology : mimicking human annotators," Physiol . Meas., vol. 33, pp. 1479, 2012. [64] N. McI ntosh, J. Becher, S. Cunningham, B. S tenson, I . A. Laing, A. J. Lyon and P. Ba dger, "Clinical d iagnosis of pneumo thorax is late: Use of trend data and decisio n support might allow preclinical detection," Pediatr. Res., vol. 48, pp. 408-4 15, 2000. [65] C. Cao, N. McIntosh, I . S. Kohane and K. Wang, "A rtifact Detection i n the PO 2 and PCO 2 Time Ser ies Monitoring Dat a from Preter m Infants," J. Clin. Monit. Com put., vol. 15, pp. 3 69-378, 1999. [66] L. Tar assenko, A. Hann a nd D. Young, "I ntegrated monitori ng and analysis for e arly warning of patient deter ioration," Br. J. Anaes th., vol. 97, pp. 64-68, 2006. [67] F. Portet, G . Feng, J. Hunter and S. Sripa da, "Evaluatio n of On -Line Bradycardia Boun dary Dete ctors from Neonatal Clin ical Data," Engineeri ng in Medicine an d Biology Societ y. 29th Ann ual International Conference of the IEEE, pp. 3288-32 91, 2007. [68] Medical Pre dictive Science Corporation, Charl ottesville , VA, USA , "HeRO Technical P ublications," 200 4 . [69] L. Chen, T . M. McKe nna, A. T. Reisner , A. Gr ibok and J. Reifman, "Decision tool for the early diagnosis of traum a patient hy povolemia," J. Biomed. Inform., vol. 41, pp. 469-478 , 2008. [70] G. D. Cliff ord, W. J. L ong, G. B. Moody and P. Sz olovits, "Robust parameter ex traction for decisio n support usi ng multimodal i ntensive care data," Philosophic al Transactions of the Royal Societ y A: Mathem atical, Physical and En gineering Sciences , vol. 367, pp. 4 11-429, 2009. [71] S. Nizam i, J. R. Gree n and C. McGreg or, "Service or iented architecture to support real- time impleme ntation of artifact dete ction in critic al care monitoring," in A nnual Internation al Conference of the IEEE En gineering in Medicine and Bi ology Society, 2011, pp. 4925-4928. [72] S. W. Ho are, D. Asbridge a nd P. C. W. Bea tty, "On-line novelty detection for arte fact identif ication in automatic anaesthesia reco rd keeping, " Med. Eng. Phys. , vol. 24, pp. 673-681 , 2002. [73] Dräger Medic al, "Instructio ns for use: I nfinity Delta Series," 2008 . [74] Dräger Medic al Syste ms Inc Telford, U.S.A ., "I nfinity gateway VF 5.0 protocol handboo k," 2007. [75] N. Shah, H. B. Ragaswamy , K. Govindugari and L. Estanol , "Performance o f three new-ge neration pulse ox imeters during motio n and low perfusion in vol unteers," J. Clin. A nesth., vol. 24, pp. 3 85-391, 8, 2012. [76] S. J. Barke r, ""Motion-resistant" pulse oximetry : A comparison of new and old model s," Anesth. Analg., vol . 95, pp. 967-972, 2002. [77] M. O'Reill y, "Reply to "'new-generation' pulse oximeters in ex tremely low-birth-we ight infants"," Journal of Perinatal and Ne onatal Nursing, vol . 26, pp. 282-283, 2 012. [78] M. Verduij n, N. Peek, N. F . de Keiz er, E. van Lieshout, A . J. M. de Pont, M. J. Schultz, E. de Jonge and B. A. J. M. de Mol, "Individual and Joint Expert Judgments as Refer ence Standards in A rtifact Dete ction," J. Am. Me d. Inform. Assoc., vo l. 15, pp. 227-234, 2008. [79] S. Cunni ngham, A. Sy mon and N. McI ntosh, "The practical manageme nt of artifact in com puterised phy siological data," Int. J. Cli n. Monit. Com put., vol. 11, pp. 211-2 16, 1994. [80] M. Blount, C . McGre gor, A. James, D. Sow , R. Kamale swaran, S. Tuuha, J. Percival and N. Pe rcival, "O n the integr ation of an artifact sy stem and a real-time heal thcare analy tics system," in IHI'10 - Pr oceedings of the 1st ACM International H ealth Inform atics Symposium, 2010, pp. 6 47-655. [81] A. Otero , P. Félix, S. Barro a nd F . Palacios, "Addressing t he flaw s of current critical al arms: a fuzz y constraint sat isfaction approac h," Artif. Intell. Med., vol. 47, pp. 219-238, 2009. [82] W. Siebe n and U. Gather , "Classify ing alarms in intens ive care - Analogy to hypothesis testi ng," Lecture Notes in Computer Science (Inclu ding Subseries Lect ure Notes in Artif icial Intellige nce and Lectur e Notes in Bioinformatics), vol . 4594 L NAI, pp. 130-138, 2007. [83] M. N. Ko stic, S. Fakhar, T . Foxall, B. S. Drak ulic and M. W . Krucoff, "Evaluation of No vel ECG Signal Processing on Quantificatio n of Transient Ischemia and B aseline Wander Su ppre ssion," En gineering in Medici ne and Biology Society, 29th Annual In ternational Con ference of the IEEE , pp. 2199- 2202, 2007. [84] J. R. Moor man, D. E. L ake and M. P. Gr iffin, "Heart rate char acteristics monitoring for neo natal sepsis," Biom edical Eng ineering, IEEE Transact ions on, vol. 53, pp. 1 26-132, 2006. [85] C. Tsien a nd J. Fackler , "An annotated d ata colle ction system to support intellige nt analysis of Intensive Care Unit dat a," Advances in Intelligent Da ta Analysis Reason ing about Data, pp. 111-121, 1997. [8 6] S . T. Lawle ss, "Crying wol f: False alarms in a pedia tric intensive care unit," Crit. Car e Med., vol. 22, pp. 981-985, 1994. [87] D. A. Sitzma n, "Executive Summary: Intellirate T echnology for Patient Monitoring," GE Me dical Systems I nformation Techn ologies, 2002. [88] A. Aboukhalil , L. Niel sen, M. Saee d, R. G. Mark and G . D. Clifford, "Reducing false alarm rates for cri tical arrhy thmias using the arte rial bloo d pressure w aveform," J. Biomed. Inf orm., vol. 41, pp. 4 42-451, 2008. [89] SAPHI RE Consortium, "I ST- 27074 SAPHI RE: I ntelligent healthcare monitoring based on a semantic interoperability platform," 20 08. [90] W. Karl en, K. Kobay ashi, J. M. A nsermino and G. A . Dumont, "Photoplethy smogram signal qu ality estimation usi ng repeated G aussian filters and cross-corr elation," Physiol . Meas., vol. 33, pp. 1617-1629, 201 2. [91] W. Karl en, C. J. Brouse, E . Cooke, J. M. Anse rmino and G. A . Dumont, "Respiratory rate estimation usi ng respiratory sinus arrhythmia fro m photoplethy smography," in Annual In ternational Conferenc e of the IEEE Engineering in Medicine and B iology Societ y, 2011, pp. 1201-1204. [92] P. Kl igfield and C. L . Green, "The cardiac safety re search consortium ECG database," J. E lectrocardiol., vol . 45, pp. 690-692, 0, 2012. RBME- 00025 -2012.R1 19       of the cardiovas cular system re lated signals," Lecture N otes in Comput er Science (Incl uding Subseries Le cture Notes i n Artificial Intel ligence and Lecture Notes in Bioinformatics), vol. 7451 LN CS, pp. 169 -170, 2012. [94] A. L . Goldberger, L . A. N. A maral, L. Glass, J. M. Haus dorff, P. C. Ivanov , R. G. Mark, J. E. Miet us, G. B. Moody , C. Peng and H . E. Stanley , "PhysioBank, P hysioToo lkit, and PhysioNe t : Components of a Ne w Rese arch Resource for Complex Phy siologic Signals," Circulat ion, vol. 101, pp. e2 15- 220, 2000. [95] A. J. E. See ly, G. C. G reen and A . Bravi, "Continuous mult iorgan variability moni toring in criticall y ill patients  complex ity science at t he bedside," in Engine ering in Medici ne and Biology S ociety,EMBC, 2 011 Annual Internati onal Conference of the IEEE, 2011, p p. 5503-5506. [96] A. Bravi, G . Green, A . Longtin and A . J. E. Seely , "Monitoring and Identification of S epsis Dev elopment throug h a Composite Me asure of Heart Rate Variabili ty," PLoS ONE, vol. 7, 2012. [97] B. Bradley , G. C. Gr een, I. Batkin an d A. J. E. Seely , "Feasibility of continuous m ultiorgan variabili ty analy sis in the intensive care unit," J. Crit. Care, vol. 27, pp. 2 18.e9-218.e20, 20 12. [98] S. Ahma d, T. Ramsay , L. Huebsch, S. F lanagan, S. McDiarmid, I . Batkin, L . McIntyre, S. R. Sund aresan, D. E. Maziak, F. M. Shamji, P. Hebe rt, D. Fergusson, A . Tinmouth an d A. J. E. Seely , "Continuous Mult i -Parameter Heart Rate Varia bility A nalysis Heralds Onse t of Sepsis in A dults," PLoS ONE, vol. 4, pp. e6 642-e6642, 2009. [99] A. Otero , P. Félix, F . Palacios, C. Pé rez-Gandía and C. O. S. Sorz ano, "Intellige nt alarms for patient s upervision," in IEEE In ternational Sym posium on Intelligent Signal Processin g, WISP, 2007, . [100] A. Oter o, P. Fél ix, S. Barro and C. Zamarró n, "A structural know ledge- based proposal for the identifi cation and chara cterization of ap noea episode s," Applied Soft C omputing Journ al, vol. 12, pp. 516-526, 2012. [101] iMDsoft Clin ical Infor mation Syste ms, " http://www.imd-soft.com/ ," l ast accessed Jan 20 13 . [102] N. Bressan , A. James and C. McGreg or, "Trends and opport unities for integrated real time neonatal cli nical decisio n support," in Biomedical a nd Health Informa tics (BHI), 201 2 IEEE-EMBS Intern ational Conference o n, 2012, pp. 687-690. [103] C. McGr egor, C. Catley , A. James and J. Pa dbury, "N ext Gener ation Neonatal Heal th Informatics w ith Artemis," Medical Inf ormatics Euro pe (MIE 2011), Oslo, Nor way, pp. 115-119, 20 11. [104] M. Blount, M. R. Ebling, J. M. Eklund, A. G . James, C. McGr egor, N. Percival, K . P. Smith and D. Sow , "Real-time analy sis for intensive care: Development an d deploy ment of the artemis a nalytic syste m," IEEE Engineering in Medicine and B iology Magazi ne, vol. 29, pp. 110-118, 20 10. [105] C. Catley , K. Smith, C. McGregor , A. James and J. M. E klund, "A framewo rk to model and transla te clinical r ules to support compl ex real-time analysis of phy siological and cli nical data," i n Proceedings of the 1st ACM International H ealth Inform at ics Sympos ium, Arlington, V irginia, USA , 2010, pp. 307-315. [106] G. F eng, S. Yaji, J. Hun ter and F . Portet, "Using Tempo ral Constraints to Integrate Signal Analy sis and Domain K nowledge in Medical Event Detection," Artif. In tell. Med., pp. 4 6-55, 2009. [107] F. Porte t, E. Reiter, A . Gatt, J. Hunte r, S. Sripada, Y. Freer and C. Sykes, "A utomatic generation of textual summ aries from neo natal intensive care data," Artif. I ntell., vol. 173, pp. 789-816, 2009. [108] A. G att and F. Portet, "T ext content and ta sk per formance in the evaluation of a Na tural L anguage Gener ation system," International Conference Rec ent Advances in Natural La nguage Process ing, RANLP, pp. 107-112, 2009. [109] A. G att, F. Portet, E. Re iter, J. Hunte r, S. Mahamood, W . Moncur and S. Sr ipada, "Fro m data to text in the neonatal intensive care Unit: Usi ng NLG technology for decision support and informatio n manageme nt," AI Communications, vol. 22, pp. 153-186, 2009. [110] J. Hunter, Y. Freer , A. Gatt, R. Lo gie, N. McIntosh, M. va n der Meulen, F. Porte t, E. Reiter, S. Sripada a nd C. Sy kes, "Summarising com plex I CU data in natural lang uage." AMIA ...Annu al Symposium Pr oceedings / AMIA Symposium.AMIA Symposium, pp. 323-327, 2008. [111] J. Hunter, Y. Freer , A. Gatt, E. Reiter, S. Sr ipada and C . Sykes, "Automatic ge neration of natural language nursi ng shift summarie s in neonatal intens ive care: BT-Nurse ," Artif. Intell. Me d., vol. 56, pp. 157-172, 2012. [112] J. R. Moo rman, C. E. Rusin, L. Hoshik, L . E. Guin, M. T . Clark, J. B. Delos, J. Kattw inkel and D. E. L ake, "Pre dictive monitoring for early detection of sub acute potentiall y catastrophic illnesses in cri tical care," in Engineering in Medicine and B iology Societ y,EMBC, 2011 An nual International C onference of th e IEEE, 2011, pp. 55 15-5518. [113] J. R. Moo rman, J. B. Delo s, A. A. Fl ower, H. Cao, B. P. K ovatchev, J. S. Richman and D. E. L ake, "Cardiovascular oscil lations at t he bedside: early diagnosis of neo natal sepsis usi ng heart rate ch aracteristics mo nitoring," Physiol. Meas., vol . 32, pp. 1821, 201 1. [114] J. R. Moo rman, W. A . Carlo, J. K attwinkel, R. L . Schelonka, P. J. Porcelli, C. T. N avarrete, E. Ba ncalari, J. L . Aschner, M. W hit Walker, J. A . Perez, C. Palme r, G. J. Stukenborg , D. E. L ake and T. Michael O'Shea, "Mortality re duction by heart rate c haracteristic monito ring in very low birth weight neonates: A randomize d trial," J. Pediatr., vol. 159, pp. 900-906.e1, 2011. [115] Y. Xiao, M . P. Griffin, D. E. Lake and J. R. Moorman, "Ne arest- Neighbor and L ogistic Regre ssion Analy ses of Clinical and Heart Rate Characteristics in the Early Diagnosis of Ne onatal Sepsis," Medical Decisi on Making, vol. 30, p p. 258-266, March/ April 2010. [116] M. P. G riffin, D. E. L ake, T. M. O'Shea a nd J. R. Moor man, "Heart rate characteristics a nd clinical signs in neonatal sepsis," Pediatr. Res., vol. 61, pp. 222-227, 2007. [117] M. P. G riffin, D. E. L ake, E. A. Bissonet te, F. E. Har rell Jr ., T. M. O'Shea and J. R. Moorman, "Hear t rate character istics: Nove l physiomarkers to predict neon atal infection a nd death," Pediatrics, vol. 116, pp. 107 0-1074, 2005. [118] B. P. K ovatchev, L . S. Farhy, H. Cao, M. P. Griffin, D . E. Lake and J. R. Moorman, "Sam ple A symmetry Analysis of Heart Rate Characte ristics with Applicat ion to Neonatal S epsis and Sy stemic Infl ammato ry Response Syndrome ," Pediatr. Res., vol. 54, p p. 892-898, print, 2003. [119] M. P. G riffin, T. M. O'She a, E. A . Bissonette, F. E. Harr ell Jr., D. E. Lake and J. R. Moorman, "A bnormal heart ra te characterist ics preceding neonatal sepsis and sepsis-like illne ss," Pediatr. Res., vol . 53, pp. 920-926, 2003. [120] D. E. L ake, J. S. Richman , M. P. Gr iffin and J. R. Moor man, "Sample entropy analy sis of neonatal hear t rate varia bility," American Journal of Physiology - Regu latory, Integrative and Comparative Physiology, vol. 283, pp. R789-R797, 0 9/01, 2002. [121] M. P. G riffin and J. R. Moorman, "Tow ard the early diagnosis of neonatal sepsis and sepsis-like illne ss using novel heart rate analy sis," Pediatrics, vol . 107, pp. 97-104, 20 01. [122] M. P. G riffin, D. F. Scoll an and J. R. Moorman , "The Dy namic Range of Neo natal Heart Rate Varia bility," J. Cardiovasc. E lectrophysiol., vol. 5, pp. 112-124, 1994. [123] P. M. Middle ton, C. H. H. T ang, G. S. H. Chan, S. Bis hop, A. V. Sav kin and N. H. L ovell, "Per ipheral photoplethysmog raphy variability analysis of sepsis patients," Me dical and Biolo gical Engineering and Computing, vol. 49, pp. 337-347, 2011. [124] A. Minutolo , M. Esposito an d G. De Pietro, "A pattern-based approach for repre senting condition-action cli nical rules into DSSs," Le cture Notes in Electrical Eng ineering, vol. 152 L NEE, pp. 777-789, 2 013. [125] D. J. Mer edith, D. Clifton, P. Charlton, J. Brooks, C. W. Pugh a nd L. Tarassenko, "Pho toplethysmo graphic derivatio n of respiratory rate: A review of relev ant physiology," Journal o f Medical Engineer ing and Technolo gy, vol. 36, pp. 1-7, 2012. [126] K. Steurbau t, K. Colpaer t, B. Gadey ne, P. Depuydt, P. Vosters, C. Danneel s, D. Benoit, J. Decruy enaere and F. De Turck, "COSA RA: Integrated service platfor m for infection surv eillance and an tibiotic manage ment in the ICU," J. Med. Sys t., vol. 36, pp. 376 5-3775, 2012. [127] A. Minutolo , M. Esposito and G. De Pie tro, "KETO: A know ledge editing tool for encoding condi tion - Action guidelines into clinical DS Ss," Lecture Notes in Computer Scie nce (Includi ng Subseries Le cture Notes in Artificial Intellige nce and Lecture Notes in Bioi nformatics), vol. 7208 L NAI, pp. 352-364, 2012. [128] N. W. Chba t, W. Chu, M. Ghosh, G. L i, M. Li, C. M. Chiofo lo, S. Vairavan, V. He rasevich and O . Gajic, "Cli nical know ledge-based inference model fo r early detection of ac ute lung injury ," Ann. Biomed. Eng., vol . 40, pp. 1131-1141, 20 12. [129] C. L . Green, P. Kl igfield, S. Georg e, I. Gussak, B. Vaj dic, P. Sager a nd M. W. Krucoff , "Detection of Q T prolongatio n using a nove l electrocardiogra phic analy sis algorithm apply ing intellige nt automation: Prospective bli nded evaluation using the Cardiac S afety Re search Consortium electrocardiogra phic database ," Am. Heart J., vol. 163, p p. 365-371, 2012. [130] B. D. Bradl ey, G. Gr een, T. Ramsay and A . J. E. Seely , "Impact of sedation and org an failure o n continuous heart and respiratory rate variability monitoring in cri tically ill patients: A pilot study ," Crit. Care Med., 2012. [131] V. Herasev ich, "I nformatics infrastruct ure for sy ndrome surveillance, decision support, reporting, an d modeling of critical illness," Mayo Clin. Proc., vo l. 85, pp. 247-25 4, 2010. RBME- 00025 -2012.R1 20 [132] K. Niemi, S . Geary , B. Quinn, M. L arrabee and K. Bro wn, "Impleme ntation and evaluation of electronic clinical decision support for compliance w ith pneumonia an d heart failure quality indicator s," American Journal of Healt h-System Pharmacy, vol. 66, pp. 3 89-397, 2009. [133] L. Al lart, C. Vilhelm, H. Mehdaoui, H. H ubert, B. Sarraz in, D. Zitoun i, M. Lemdani and P. Ravaux, "A n architecture fo r online compariso n and validation of proce ssing metho ds and computeriz ed guidelines i n intensive care units," Comp ut. Methods Progr ams Biome d., vol. 93, pp. 93-103, 2009 . [134] F. W u, M. Williams, P. K azanzides, K . Brady and J. F ackler, "A modular clinical decision support system cl inical prototy pe extensible into multiple clinical se ttings," in 3rd I nternational Conference on Per vasive Computing Tech nologies for H ealthcare , 2009, pp. 1-4. [135] E. Gr onvall, L. Piccini, A . Pollini, A. Rul lo and G. A ndreoni, "Assemblie s of Heterogene ous Technologies at the N eonatal I ntensive Care Unit," Ambient I ntelligence, pp. 340- 357, 2007. [136] X. Hu, M. Sa po, V. Nenov , T. Barry , S. Kim, D. H. Do, N. Boy le and N. Martin, "Pre dictive combinatio ns of monitor alarms prece ding in-h ospital code blue eve nts," J. Biomed. Inf orm., vol. 45, pp. 913-921, 2012. [137] X. Sun, P . Yang, Y. L i, Z. Gao and Y. T . Zhang, "Robus t heart beat detection from photo plethy smography interlace d with motion artifa cts based on empirical mo de decomposit ion," Proceedings - IEE E-EMBS Interna tional Conference on Bi omedical and H ealth Inf ormatics: Global Gra nd Challenge of Health Inform atics, BHI 2012, pp. 775-778, 2012 . [138] M. Borow ski, S. Siebig, C. Wrede and M . Imhoff , "Reducing false alarms of intens ive care online-monit oring syste ms: An evaluation of two signal extractio n algorithms," Compu tational and M athematical M ethods in Medicine, 2011. [139] S. Char bonnier and S. Ge ntil, "On-line adaptive trend extraction of multiple physio logical signals for al arm filter ing in intensive care units," Int J Adapt Control Signal Process, vol. 24, pp. 382-408, 201 0. [140] S. Char bonnier, G. Becq a nd L. Biot, "O n -line se gmentation algo rithm for continuously monitored data in intensive care units," Biomedical Engineering, IEE E Transactio ns on, vol. 51, pp. 48 4-492, 2004. [141] J. M. Blum, G . H. K ruger, K. L. Sanders, J. G utierrez and A. L. Rosenberg, "Spe cificity improv ement for netw ork distributed phy siologic alarms based on a simple dete rministic react ive intellig ent agent in the criti cal care environme nt," J. Clin. Monit. Comput., vol. 23, pp. 21-30, 2009. [142] W. Zo ng, G. Moody an d R. Mark, "Reduct ion of false arter ial blood pressure alarms u sing signal quali ty assesse ment and relationships betwee n the electro cardiogram and arter ial blood pressure ," Medical and Biologic al Engineering an d Computing, vol. 42, pp. 698-706, 2004. [143] S. Jakob, I . Korhonen, E. Ruokone n, T. Vi rtanen, A . Kogan and J. Takala, "Detect ion of artifacts in monitore d trends in intens ive care," Comput. Methods Programs Biomed., vol. 63, pp. 203-209, 2000. [144] Y. Zhang a nd P. Szolo vits, "Patient-specific le arning in real time for adaptive monitor ing in critical care ," J. Biomed. I nform., vol. 41, pp. 452-460, 2008. [145] C. L . Tsien, "Event discov ery in medical t ime-series data." in Proc AMIA Symp. 2000, p p. 858-862. [146] C. L . Tsien and J. C. Fac kler, "Poo r prognosis for e xisting monitors in the intensive care unit," Crit. Care M ed., vol. 25, pp. 614-619, 1997. [147] V. Monasterio , F. Burgess a nd G . D. Clifford, "Robust class ification of neonatal apnoe a-related desaturatio ns," Physiol. Meas ., vol. 33, pp. 1503, 2012. [148] H. Le e, C. G. Rusin, D. E. L ake, M. T. Clark, L . Guin, T. J. Smo ot, A. O. Paget-Brow n, B. D. Vergal es, J. Kattwinkel, J. R. Moo rman and J. B . Delos, "A new algorithm for de tecting central apnea in neona tes," Physiol. Meas., vol. 33, pp. 1, 2012. [149] S. Y. Be lal, A. J. Emmer son and P. C. W. Beat ty, "A utomatic detection of apnoea of pre maturity," Physiol. M eas., vol. 32, pp. 523, 2011. [150] J. A. Qui nn, C. K. I . Williams and N. McIntosh, "Factori al Switching Linear Dy namical Systems A pplied to Phy siological Conditio n Monitoring," Pattern Analysis and Machine I ntelligence, IEEE Transacti ons on, vol. 31, pp. 1537-1551, 20 09. [151] C. L . Tsien, I. S. Ko hane and N. McI ntosh, "Building I CU artifact detection model s with more data in less time ." in Proc AMIA Symp. 200 1, pp. 706-710. [152] C. L . Tsien, I. S. Ko hane and N. McI ntosh, "Multiple signal integr ation by decision tree induction to de tect artifacts in t he neonatal inten sive care unit," Artif. Intell . Med., vol. 19, pp. 189-202, 2000. [153] M. Koe ny, S. Leonhardt, M . Czaplik an d R. Rossaint, "O n the road to predictive smar t alarms based on a netw orked operating room," Proceedings - IEEE Symposium on Comput er-Based Medical Systems, 2012 . [154] P. Yang, G . Dumont and J . Ansermino, "Se nsor Fusion Using a Hy brid Median Fil ter for Artifact Remo val in Intraope rative Heart Rate Monitoring," J. Clin. Monit. Com put., vol. 23, pp. 75-83, 2009. [155] J. M. A nsermino, J. P. Da niels, R. T. He wgill, J. L im, P. Yang, C. J. Brouse, G. A . Dumont and J. B. Bowering, "A n evaluation of a nov el software tool for detecting changes in p hysiological mon itoring," Anesth. Analg. , vol. 108, pp. 873-880, 2009. [156] R. K. G ostt, G. D. Rath bone and A. P. T ucker, "Real-time pulse oximetry artifac t annotation on com puterized anaest hetic records," J. Cl in. Monit. Comput., vol . 17, pp. 249-257, 2002. [157] D. F. Sittig and M. Facto r, "Physiol ogic trend detection and artifact rejection: a par allel impleme ntation of a multi-state K alman filtering algorithm," Comp ut. Methods Progr ams Biomed., vol . 31, pp. 1-10, 1990. [158] M. J. Nava bi, K. C. My lrea and R. C. W att, "Detection of f alse alarms using an integ rated anesthesia mo nitor," in Engineering i n Medicine and Biology Society, I nternational C onference of the IEEE , 1989, pp. 1774-17 75. [159] A. T. Reis ner, L . Chen and J. Reifman, "T he association between vital signs and major hemorrhagic i njury is significantl y improve d after controll ing for sources o f measureme nt variability ," J. Crit. Care, vol. 27, pp. 533.e 1- 533.e10, 2012 . [160] F. J. Mar tinez-Tabares, J. Es pinosa-Oviedo and G . Castell anos- Dominguez, "I mprove ment of ECG signal quali ty measure ment using correl ation and diversity-based appro aches," Proceedin gs of the Annu al International C onference of th e IEEE Engine ering in Medicine and Bio logy Society, EMBS, pp. 4295-4298, 2012. [161] I . Silva, J. Lee and R. G. Mark, "Signal quali ty estimation w ith multichannel adap tive filtering in intensive care settings," IEEE Transac tions on Biomedical En gineering, vol. 59, pp. 2476-2485, 2012. [162] M. Bsoul, H . Minn and L . Tamil, "A pnea MedAssist: Real-time Sl eep Apnea Monitor Us ing Single-Le ad ECG," Informati on Technology in Biomedicine, I EEE Transactions on, vol. 15, pp. 416-427, 2011. [163] Henian Xia, G . A. G arcia, J. C. McBride, A. Sull ivan, T. De Bock , J. Bains, D. C. W ortham and Xiaope ng Zhao, "Comp uter algor ithms for evaluating the q uality of ECGs in real time," Computin g in Cardiolo gy, pp. 369-372, 2011 . [164] S. Zaunse der, R. Huhle and H. Malberg , "CinC chall enge - Assessing the usability of ECG by ensemble decision tree s," Computers in Cardiol ogy, vol. 38, pp. 277-2 80, 2011.       driven approac h to ECG signal q uality assessment using multis tep SVM classification," Comp uters in Cardi ology, vol. 38, pp. 453-45 5, 2011. [166] U. R. A charya, E. C. Chua, O. F aust, T. Lim, L . Feng and B. Lim, "Automated detection of sle ep apnea fro m electrocardiogram signals us ing nonlinear parame ters," Physiol. Me as., vol. 32, pp. 287, 2 011. [167] A. K handoker and M. Pala niswami, "Mo deling Respiratory Movement Signals During Ce ntral and Obs tructive Sleep Apnea Eve nts Using Electrocardiogram," Annals of Biome dical Engine ering, vol. 39, pp. 80 1-811, 2011. [168] S. Nizami, J. R. Gree n, J. M. Eklund an d C. McGre gor, "Heart disease classification t hrough HRV anal ysis using Parall el Cascade I dentification and Fast Orthogonal Se arch," Medical M easurements a nd Applications Proceedings (MeM eA), IEEE I nternational Wor kshop on, pp. 134-139, 2 010. [169] S. Nemati , A. Malhotra and G . D. Cliffo rd, "Data fusion fo r improve d respiration rate e stimation," EURASIP J. Adv. Sign al Process, pp. 1-10, 2 010.        , R. Hornero, J. V . Marcos and F . del Campo, " Multivariate Analysis of Blood Oxyge n Saturation Reco rdings in Obstr uctive Slee p Apnea Diagnosis," Biomed ical Engineer ing, IEEE Tr ansactions on, vol. 57, pp. 2816-2824, 2010. [171] E. Gil , J. María Vergara a nd P. L aguna, "Detection of decre ases in the amplitude fluc tuation of pulse p hotoplethy smography signal as indication of obstructive sl eep apnea syndrome in children," Biomedical Signal Pro cessing and Control, vol . 3, pp. 267-277, 20 08. [172] E. Gil, M. Me ndez, J. M. Ver gara, S. Cerutti, A . M. Bianchi a nd P. Laguna, "De tection of obstructive slee p apnea in children using decre ases in amplitude fluc tuations of PPG signal and HRV," Engineer ing in Medici ne and Biology Society, I nternational C onference of the IEEE, pp. 3479-3482, 200 8. [173] C. Yu, Z. L iu, T. McKe nna, A. T . Reisner and J. Reifma n, "A Met hod for Automatic I dentification of Reliable Heart Rate s Calculated fro m ECG and PPG W aveforms," Journal of the American Me dical Informati cs Association, vol . 13, pp. 309-320, 200 6. [174] I ntelligent Syste ms Group, "TRACE, T ool foR anAly zing and disCovering patt Erns, https://www .gsi.dec.usc.e s/trace ," Department of Electronics an d Computer Scien ce, Univers ity of Santiago D e Compostel a, La Coruña, Spain.