LabelECG: A Web-based Tool for Distributed Electrocardiogram Annotation

LabelECG: A Web- based Tool for D istributed Electrocardiogram Annotation Zijian Ding 1 , Shan Qiu 1 , Yutong Guo 2 ,Jianping Lin 3, 4 , Li Sun 3 , Dapeng Fu 5 , Zhen Yang 6 , Chengquan Li 4 , Yang Yu 7 , Long Meng 8 , Tingting Lv 4, 9 , Dan Li 9 and P ing Zhang 9, 4* 1 Department of Electronic Engineering , Tsinghua University , Beijing, China 2 School of Information and Electronics, Beijing Institute of Technology, Beijing, China 3 Xinheyidian Co. Ltd , Beijing, China 4 School of clinical Medicine, Tsinghua University, Beijing, China 5 Chinese Academy of Sciences Zhong Guan Cun Hospital, Beijing, China 6 ECG Center, Tianjin Wuqing District People's Hospital, Tianjin, China 7 The Affiliated Hospital of Qingdao University, Qingdao, China 8 Shandong Mingjia technology Co., Ltd, Taian, China 9 Department of Cardiology, Beijing Tsinghua Changgung Hospital , Beijing, China zhpdoc@126.com Abstract. Electrocardiography pla ys an essential role in diagnosing and scree n- ing cardiovascular disease s in daily healthcare. Deep neural netw orks have shown the potentials to improve the accuracies of arrhythmia detection based on electrocardiograms (EC Gs). However, more EC G records with groun d truth are needed to pr omote the development and progression of deep learning tec h- niques in automatic ECG analysis. Here we propo se a web-based tool for ECG viewing and anno ta ting, LabelECG . With th e facilitation of unified da ta ma n- agement, LabelECG is able to distribute large cohorts of ECGs to dozens of technicians an d physicians, who can simultaneously make annotations through web-browsers on PCs, tablets and cell phones. Along with the doctors from four hospitals in China , we applied LabelECG to sup port the annotations of about 15 ,000 12 -lead restin g ECG records in three months . These annotated ECGs have successf ul ly sup ported the First China ECG intelligent Competition. L a- belECG will be freely accessible on the Internet to support similar researches, and will also be upgraded through future works. Keywords: Cardiovascular disease, Electrocardiograms , distributed annotation. 1 Introduction Electrocardio graphy is a com mon ap proach to diagnose and screen cardiovascular diseases in clinic. Due to its characteristics incl uding non -invasive, eas y- to -operate and econo mical, it ’ s the most widely ad opted clinical detectio n to diagnose ar rhyt h- mia , myocardial isc hemia and myocardial in farction [1 ] . In order to dea l with large amounts of electrocardio grams (ECGs ), co mputerized i nterpretations ai m to i mprove 2 the co rrectness of ECG dia gnose and alleviate the workload s of p hysicians [2] . Though there ar e dozens of c omputerized interpr etation syste ms for ECGs, e.g. GE Marquette system [ 3] and Glasgo w system [4] , co mputerized interpr etation are still suffered from the limited d iagnostic accuracies [2] . As the fast gro wth and huge success of deep neural networks i n co mputer vision and na tural language p rocessing, etc. [5] , these co mpu tational techniques are expected to impact the area of precision car diovascular medicine, including auto matic ECG interpretation [ 6] . Recen tly H annun et al p ublished their work on detecting and class i- fying arrh ythmia based on single-lead ECG dat a [7 ] . T he deep ResNet net work achieved better results compared to several technicians, which was trai ned on almost 91 ,232 r ecords and tested on 328 records fro m unique p atients. Similarly, Attia et al reported that a neural netw ork, trained on 35,9 70 ECG-echoca rdiogram pair s and tested on 52,7 80 ECG reco rds, can screen for asymptomatic left ventricular dysfun c- tion [8]. T hese works s how that dee p neural net works are ab le to improve the dia g- no stic acc uracies based on large collection s of ECG reco rds. However, almost all ECG databases with ca reful an notations ar e s mall in sample sizes, whic h might i nhibit the application and progressio n of deep neural net works. The applicatio n of an image annotation tool na med LabelMe [9] laid the foundation of the well-known I mageNet dataset [10] that flourish the research of deep learning. Similarly, a n EC G an notation tool can help build lar ge co llections o f ECG rec ords with ground truth. As a result, ECG databases with large sa mple size s will p romote the research of deep learni ng in the co mputerized anal ysis. In th is pap er, we prese nt a web -based tool for distributed E CG annotatio n named LabelECG. P hysicians and te chnicians ca n annota te E CG records with various ti me lengths and number o f lead s, through the web -browsers on d esktops, lap tops, tablets and cell phones at a nytime and anywhere. What’s more, ECG datasets are under un i- fied management, and can b e accessed by se veral d octors simultaneousl y . Four do c- tors used LabelECG to annotate almost 15 ,000 12-lead r esting ECG records. The resulting database has supported the First China ECG AI Challenge [11]. LabelECG will be accessible onli ne to sup port similar researches. 2 Related Work ECG annotati ng often requires expert kno wledge and laborio us work. Considering the tedious clinical workload of d octors , the manipulations should take up less time and improve efficiency. For exa mple, the tool should provide a convenient way to access such that docto rs can use spare ti me to annotate d ata . What’s more, the tool should be responsible for data management such that docto rs can focus o n the data an notation. However, most previo us tools fail to ful fill these require ments. Most tools ar e used off-line (see T able 1). As a result, computers with installed tools and u ser manuals should b e pr ovided for doctors, who ha ve to spend time to learn and use these tools. Amo ng all tools sho wn in T able 1, WaveformECG [16 ] is the only web-based tool . Ho wever due to unk nown reaso ns, the tool is not accessible 3 currently. For most off -line tools, the pro blem is t hat doct ors have to b e resp onsi ble for data management. The major d ifference bet ween LabelECG and these previo us too ls is on t w o a s- pects. First of all, L abelECG is a web-based tool for distributed ECG annotation. Several do ctors can access LabelECG through web -browsers on desktops, laptops, tablets and even cell phones. Compared to WaveformECG, LabelECG i s easier to deploy since it ’ s based on docker. As a r esult, LabelECG is more suitab le when do c- tors cannot upload d ata to the I nternet a nd have to make annotations i n a local area network. Seco ndly, LabelECG is responsib le for d ata management . Doctors ca n i g- nore the manipulation o f data and focus on ECG an notation. Ta ble 1. Tools for ECG annotation. Name Access Functions SigViewer [12] Off-line Multi-lead viewing, diagnose annotating EcgEditor [13] Off-line Multi-lead viewing, QRS detection, diagnose annotating ECG Viewer [14] Off-line QRS detection, diagnose annotating BSS_ECG [15] Off-line QRS detection, diagnose annotating WaveformECG [16] Online Multi-lead viewing, QRS detection, diagnose annotating Fig. 1 . The organization of LabelECG. Multiple doctors can upload, visualize, annotate and review ECG record s through the frontend s. The backend support these functions through three databases and two servers. LabelECG can be deplo ye d on any clo ud systems to connect the frontend and the backend. 3 LabelECG LabelECG i s a web-based tool for distrib uted ECG annotati on ( see Fig. 1). Through the web b rowsers of desktops, laptops, tablets and cell phones, multiple doctor s can 4 collaborate on annotating the diagnoses of ECG records at any time and any place . With the help of unified data management, docto rs are ab le to focus on annotatio n without manipulating hundr eds of ECG records. The distributed system consists of a fronte nd, a backe nd and a communicating cloud server. T o explicitly explain the usage an d deplo yment of LabelECG, we intr o- duce the functions and architecture in t he follo wing two sections. Firstly, we mai nly discuss how to login LabelECG, cho ose a dataset, visualize and annotate an ECG record, and revise the perso nal annotated record s. Seco nd ly, we mainly disc uss how LabelECG is organized to support the above mentioned functions. Fig. 2. The functions of LabelECG. (A) Register and login; (B) Select a d ataset to ECG ann o- tating; (C) Visualize and annotate diagnoses; (D) Personal accounts . 3.1 Functions LabelECG is able to help u sers upload, visualize, an notate and revise their EC G re c- ords. To make these manipulations, the functions of LabelECG is desig ned to include four parts, including making registration and login, choosin g a dataset, annota ting the diagnoses, and revising all p ersonal annotations (see Fig. 2) . 5 First of all, the establishment of personal accounts makes it p ossible to track all annotations of each user . One character is that in co nsideration of d ata security, a system administrator needs to provide a verificatio n co de to each user to complete registration. Th is manipulation aims to ensure that o nly t he specific users can ha ve access to their ECG data. Secondly, LabelECG offer s almost all open source datasets from P hysionet [ 17], as well as the user uploaded datasets. Users can choo se their own d ataset and begin ECG annotating. O ne characteristic of LabelECG is t hat when entering the dataset, users can begin with t he last record in their last or previous annotatio n. LabelECG intr o- duces the Lightwave system [17] to visualize any E CG r ecords with various time lengths and number of lead s. Meanwhile u sers can hide spe cific leads in order to f a- ci litate the observation o f certain leads. Three d ialog boxes above the vis ualizatio n are used to help make annotations: the box on the left side pr ovides ECG parameters s uch heart rate, the one in the mi ddle provides auto matic dia gnoses, and the o ne on the right is for writing annotation s. Another characteristic o f LabelECG is that users can label one record as either “ confirmed ” or “ unsure ” , since some EC G record s may b e too ambiguous to an notate. Furthermore, as multiple u sers can collabo rate on annotatin g one dataset, L a- belECG gives ri ghts to advanced and experie nced experts to v erify all a nnotations among the se users. B esides, LabelECG enhances intra-group co mmunication b y ma k- ing the unsure ECG data visible for all group members. Fig. 3. The architecture o f LabelECG. A user management system, an ann otation management system and the Lightwave system support all the functions of LabelECG . 3.2 Architecture To support the ab ove mentioned f unctions, LabelEC G is built upon three systems, including a user manageme nt syste m, a n annotation management syste m and a Lightwave syste m (see Fig. 3 ). T he user management system supports the function of user regi stration a nd user infor mation management. The annotation manageme nt sys- 6 tem supports the f unction of up loading and storing ECG data and their corresponding parameters, auto matic diag noses and user annotations. The Lightwave s ystem su p- ports the function of ECG vi sualization. User M anagement Syste m. Th is syste m mainly supports the function of user regi s- tration and user i nformation st orage . It includes a web page for registration and log in, a user in formation d atabase to store perso nal accou nts, and a server to connect the front web page and th e user in formation database. To be specific, users need to register their acco unts for ECG an notation. We have set “ Password ” etc. as the r equired information. After registration, users can lo gin to enter their acc ounts. T he log-in web page sends a login request to the server, and it will c heck the input i nformation. It will re flect a succes sful co nnected p rompt i f the user’s i nformation is found. Otherwise, it will a sk the user to check the fill-in in fo r- mation . T he fron tend o f LabelECG offers an online working environment on compu t- ers, tablets, o r cell phones. T he backend ser ver uses the framework of node.js and Express as the Co mmon Gat e w ay Interface (CGI). By keep ing abreast with t he r e- quest fro m the front end, the bac k end will retur n the co rresponding data and info r- mation. Annotation M anage m ent Syste m . Th is syste m mainly supp orts the fu nction o f a n- notating a nd revising ECG record s. It in cludes an annotation web page, a d atabase to store ECG data and corresponding par ameters, automatic diagnoses and user an not a- tions, and a ser ver to con nect the fronte nd an notation web p age with t he backend database. To be specific, if users have their ECG data to annotate, there exists an interface to transfer data into t he backe nd ECG signal datab ase. There is a backe nd server to d e- code and transfer the raw E CG data into the form fitting the standard of LabelECG. Users are also able to view ECG d ata from Phys ionet [17] . After enteri ng the an not a- tion page, the first-time users need to choose the first data to annotate . Af ter the first - time annotation, the s ystem automaticall y shows t he record next to the ones they pr e- viously annotated . In the annotatio n web page, users can visualize the ir ECG data and make annot a- tions. We designed a “ confir m ” button and a n “ Unsure ” button for a nnotation. After writing annotations into the right side box and pressing the “ confirm ” button, the server sends th is message into the client’s li st of the Dia gnosis Info dat ab ase. If users press the “ Unsure ” button, this particular data is stored into a p articular list in the Diagnosis Info d atabase. Once users have clicked on one of the mentioned buttons, the interface would auto maticall y t urn to the next reco rd. Mo reover, we design "Next One" or "P revious One" button for users to vie w the nearby data. In o rder to review t he p ersonal annotations , u sers ca n p ress the " Account" button and enter the re view patter n. As for regular users, t he label ed ECG data and an not a- tion are shown i n order . T hey can clic k on the data number and enter t he annotation page to make revises. If a user ha s a n e xpert account , s/ he c an also c heck other users’ annotations. 7 Lightwave system. T his syste m mainly suppo rts the f unction o f ECG d ata visualiz a- tion. P hysionet provides this system online [17]. We run this s ys tem as a CGI applic a- tion . Once the front end sen ds requests to the b ack e nd, a web ser ver collects a nd forwards t hem to t he Lightwave s ystem. Afterwards, the Light wave system will parse the requests and access to the correspo nding database in order to o btain data. 4 Supporting the First China ECG Intelligent Competition The First China ECG Intellig ent Co mpetitio n ai ms to e ncourage t he d evelop m ent of algorithm s to classify, fro m 12 -lead resting ECGs with various time lengths, whet her an ECG record s hows nor mal, atrial fibrillation, early rep olarization and T wave change, etc [11] . In order to ensure the data qualit y, d octors who co me from Beij ing Tsinghua Changgung hospital , Chinese Acade my of Sciences Zhong Gua n Cun Ho s- pital, Tianj in W uqing Di strict Peo ple's Ho spital, and the Af filiated Ho spital of Qin g- dao University, used LabelE CG to visualize, annotate and revie w ab out 15 ,000 r e c- ords. LabelECG helped pair four doctors as two tea ms, a nd gave r ights to two e xper i- enced doctor s to r eview and revise all an notated records. With the assista nces of L a- belECG, these doctors finishe d annotations in ab out three months which guaran teed the success of the co mpetition. 5 Discussion LabelECG i s a web-based tool for distrib uted ECG a nnotation. Multiple d octors can upload, visualize, an notate and revise E CG records via web browsers through des k- tops, lapto ps, tablets and cell phones. LabelECG is ab le to distribute one d ataset to several doctors for collaborative annotation. It is also responsible for unified data management such that doctors can focus on data annotati on . With docto rs as first users, LabelECG suppo rted the First China ECG Intellige nt Competitio n. Our doctors annotated about 15, 000 12-lead resting ECG record s in about three months. The current version of LabelE CG can make annotatio ns o n diagnoses but lacks the ability to annotate local in formation such as beats and waves . We will add functions including b eats annotat ion and fiducial point annotation, and add m ore auto matic analysis functions, in order to reduce burden and improve efficiency. I n addition , since the current v ersion supported the Co mpetition, La belECG w as prese nted in Chinese. We will distribute this version of LabelECG in English. Acknowledgement This work is supported by The National Key Research a nd Development Progra m of China (2017YFB 1401804) . 8 Re ferences 1. Clifford, G.D., Azuaje, F. and McSh arry, P. : Advanced methods and tools for ECG data analysis. Artech house , Boston (2006). 2. Schlä pfer J, Wellens, HJ: Computer-interpreted electrocardiograms: benefits an d limit a- tions. Journal of the Ame rican College of Cardiology, 70(9), 1183-1192. (2017). 3. Marquette 12SL ECG A nalysis Program, Statement of Validation and Accuracy, http://gehealthcare. com, last accessed 2019/7/13. 4. Macfarlane P W, Devine B, Clark E: The university o f Glasgow (Uni -G) ECG analysis program. In Computers in Cardiology, 2005, (pp. 451-454). IEEE, (2016). 5. LeCun Y, Bengio Y, Hinton G: Deep learning. nature, 521(7553), 436 (2015). 6. Krittanawong C, Zh ang H, Wang Z, Aydar M, Kitai T: Artificial intelligence in precision cardiovascular medicine. Journ al of the American College of Cardio logy. 22 ; 6 9(21):2657- 64 (2017). 7. Hannun AY, Rajpurkar P, Haghpanahi M, Tison GH, Bourn C, Turakhia MP, Ng AY: Cardiologist-level arrhythmia d etection and classification in ambulatory electrocardi o- grams using a deep neural network. Nature m edicine. 25(1):65 (2019). 8. Attia ZI, Kapa S, Lopez -Jimenez F, McKie P M, Ladewig DJ, Sata m G, Pellikka PA, E n- riquez-Sarano M, Nosew orthy PA, Mu nger TM, Asirvatham SJ : Screening for cardiac co ntractile dysfunction using an artificial intelligence – enabled electrocardiogram . Nature medicine 25(1):70 (2019). 9. Russell BC, To rralba A, Murph y KP, Freeman WT: LabelMe: a database and web-based tool for image annotation. International journ al of computer vision , 1;77(1-3):157- 73 (2008). 10. Deng J, Dong W, Socher R, Li LJ, Li K, Fei -Fei L. Imagenet: A large-scale hierarchical image database. In20 09 IEEE conference on computer vision an d pattern recognition , p p. 248 -255 (2009). 11. The First China ECG Intelligent Competition, http ://m di.ids.tsinghua.edu.cn/#/, last a c- cessed 2019/7/13. 12. Lin Y, Brunner C, Sajda P, Faller J. SigViewer: Visualizing Multimodal Signals Stored in 13. EcgEditor, https://github.com/Unisens/EcgEditor , last access ed 2019/7/13. 14. ECG Viewer, htt ps://github.com/jram shur/ECG_Viewer, last accessed 2019 /7/13. 15. BSS_ECG, h ttps://github.com /AdnanHidic/bss_ecg, last accessed 2019/7/13. 16. Winslow R L, Granite S, Jurado C. WaveformECG: A P latform for Visualizing, Annota t- ing, and Analyzing ECG Data. Com pu ting in science & engineering , 26;18(5):36 (2016). 17. Goldberger AL, Amaral LA, Glass L, Hausdorff JM, Ivanov PC, Mark RG, Mietus JE , Moody GB, Peng CK, S tanley HE : P hysioBank, P hysioToolkit, and PhysioNet : comp o- nents of a new research re source for complex p hysiolo gic signals. Circu lation , 101(23):e215- 20 (2000).