Low congestion online routing and an improved mistake bound for online prediction of graph labeling

Lo w congestion online routing and a n impro v ed mistak e b ound for online prediction of graph lab eling Jittat F ak c haro enphol ∗ Bo onserm Kijsirikul † Abstract In this pap er, we show a connection b etw een a certain online low-congestion routing problem and an online prediction of graph labeling. More sp ecifically , we prov e tha t if ther e exists a routing scheme that guarantees a congestion o f α on any edge, there exis ts an online prediction algorithm with mista ke bound α times the cut s iz e, whic h is the size of the cut induced by the lab el partitioning of graph vertices. Wit h previous known b ound of O (log n ) for α f or t he ro uting problem o n trees with n v ertices, we obtain an improv ed prediction algorithm for graphs with high effective resistance. In c o ntrast to previous approa ches that mov e the g r aph pr oblem in to problems in vector space using gra ph L a placian and rely on the analysis of the p erceptron a lgorithm, our pro of are purely combinatorial. F urther more, our appr o ach directly generalizes to the case where lab els are no t binary . 1 In tro duction W e are interested in an online p rediction p roblem on graphs. Giv en a connected grap h G = ( V , E ) and a lab eling ℓ : V → {− 1 , +1 } , un kno wn to the p r ediction algorithm, in eac h round i , for i = 1 , 2 , . . . , an adversary asks for a lab el of a v ertex v i ∈ V , the p r ediction algorithm pro vides the answer y i , and then receiv es th e correct lab el ˆ y i = ℓ ( v i ). The goal is to minimize the n um b er of rounds th at the algorithm mak es a mistak e, i.e., r ounds i suc h that y i 6 = ˆ y i . T o m ake our present ation clean, in this w ork we do not coun t the mistake made on the first question v 1 . 1 This pr oblem has b een studied with standard online learning to ols such as the p erceptron algorithm. Herbster, P ont il, and W ainer [6], and Herbs ter and Po ntil [5] u se ps eu doin v erse of graph Laplacian as a kernel and pr o vid e a mistak e b oun d that dep end s on the size of the cut induced b y the partition based on th e real lab eling of vertic es and the largest effectiv e resistance b et we en any p air of v ertices in the graph. Recently , Herbster [4] exploits the cluster structure of th e lab eling on the graph, and pro vides an improv ed mistak e b ound s. P elc km ans and Su yk ens [7] presen t a com binatorial algorithm for the problem that predicts a lab el of a give n v ertex based on k n o wn lab els of its neigh b ors. They also pro v e a b ound on the n um b er of mistak es when the la b els of adj acen t vertices are known. Ho wev er, their b ound is very lo ose since it do es not count ev ery mistak es and their pro of is still b ased on graph Laplacian. W e shall compare the b oun d that we obtain with p revious b ounds of Herbster et. al. [6 , 5, 4] and of Pelc kmans and S uyk ens [7] in S ection 3.1. ∗ Department of Computer Engineering, Kasetsart U niversi ty , Ba ngkok, Thaila nd 10900. E-mail: jittat@gma il.com . Supp orted by the Thailand R esearc h F und Grant MRG508 0318. † Department of Computer Engineering, Chulalo ngkorn Universit y , B angkok, Thaila nd 10330. E-mail: Boonserm.K @chula.ac.th . 1 T o p rop erly account this, one can simply add 1 to our mistak e b oun d. This work follo ws th e initiation of P elc km ans and Suykens. W e show connection b et we en the prediction problem and the follo wing online routing problem, first in tro duced by Awerbuc h and Azar [1 ] in th eir study of online multica st r outing. Given a connected graph G = ( V , E ), the algorithm receiv es a sequence of requests r 1 , r 2 , . . . , wh ere r i ∈ V , and, for eac h r i , where i > 0, has to route one unit of flo w from r i to some pr evious know r j where j < i . T he algo rithm w orks in an online fashion, i.e., it has to r eturn a route f or r i b efore receiving other requests r i ′ , where i ′ > i . Given a set of r outes, w e define the c ongestion C ong ( e ) incurred on edge e ∈ E , defined as the n um b er of routes th at use e . The p erform ance of the algorithm is measured by the maxim um conge stion incurred on an y edge. W e p ro v e, in S ection 2, t hat if th ere exists an algorithm A with a guarante e th at the congestion in cu rred on an y ed ge will b e n o greater than α , there exists an online prediction algorithm with the mistake b ound of α · | cut ( ℓ ) | , where cut ( ℓ ) b e the set of edges joining pairs of v ertices with differen t labels, i.e., cut ( ℓ ) = { ( u, v ) ∈ E : ℓ ( u ) 6 = ℓ ( v ) } . In Section 3, w e apply th e kno wn congestion b ound to sh o w the mistake b ound f or th e graph prediction problem, and compare the b ound obtained with the b ounds from previous resu lts. W e note that our approac h directly generalizes to the case when lab els are not binary (i.e., when the lab eling fun ction ℓ maps V to an arb itrary set L of lab els) with th e same mistake b ound . 2 Reduction to lo w-congestion routing W e first p resen t an online p rediction algorithm from an online routing algorithm A . The pre- diction algo rithm P A is v ery simple, giv en a v ertex v i , it u ses A to r oute one unit of fl o w from v i to an y v ertices v j with kno wn lab els, it then returns the kno wn lab el ℓ ( v j ) as the prediction. W e prov e the follo wing theorem. Theorem 1 If A guar ante es that no e dges is use d mor e than α times, the pr e diction algorithm would make at most α · | cut ( ℓ ) | mistakes, not including the mistake made on the first qu e ry v 1 . Pro of: W e shall show that the num b er of mistak e is at m ost α · | cut ( ℓ ) | . No te that for eac h mistak e P A mak es on v ertex v i , A routes v i to some kn own vertex v j along a path P i . Since P A predicts ℓ ( v j ) and mak es a mistak e, w e hav e ℓ ( v i ) 6 = ℓ ( v j ); thus, P i m ust use some cut edge e in cut ( ℓ ). W e charge this mistake to e . W e note that P i ma y use many cut edges, b ut we only c harge the mistak e to one arbitrary edge . Since the routing pro du ced by A u ses eac h ed ge no more than α times, eac h cut edge is c h arged no more than α times as w ell. Th er efore, the n umb er of mistak es P A mak es m ust b e at most α · | cut ( ℓ ) | , as required. W e note that this pro of do es not use any fact that the lab eling ℓ is bin ary; therefore, the pro of holds for general lab eling as w ell. 3 Mistak e b ound T o obtain the mistak e b oun d , we first state the result on the online routing on trees. The theorem b elo w fir st app eared in the wo rk of Awerbuc h and Azar [1], in w hic h they called the problem r estricte d offline multic ast , and has b een disco vered indep endently b y Chalermso ok and F ak c haro enp hol [2]. W e state the result in the form in [2] as it matc hes our settings. Theorem 2 (Theorem 4.4 in [1], Theorem 1 in [2 ]) F or any tr e e T with n vertic es and any se quenc e of vertic e s t 1 , t 2 , . . . t k in T , ther e exists an efficient algorithm that finds a set of p aths q 1 , q 2 , . . . , q k − 1 such that (1) q i c onne cts t i +1 to some t j , suc h that j ≤ i , and (2) e ach e dge in T b e longs to at most O (log n ) p aths. Mor e over the p ath q i dep e nds only on p aths q 1 , q 2 , . . . , q i − 1 . W e note that the b ound also holds for general graph G b y taking T to b e its spanning tree. Using Theorems 1 and 2, w e obtain the f ollo wing mistak e b ound . Theorem 3 F or gr aph G = ( V , E ) and an unknown la b eling ℓ : V → L , ther e exists an efficient pr e diction algorithm that makes at most O (log | V | ) · | cut ( ℓ ) | mistakes, wher e cut ( ℓ ) denotes the set of e dges joining p airs of vertic es with differ ent lab els. W e note that for line g raph , our algorithm is o ptimal. One can pro ve, in the same w ay as the pro of of optimalit y of b inary searc h , that an adv ersary can fo ol any alg orithm to make Ω(log n ) mistak es on a line. 3.1 Comparison to previous bounds W e compare our mistak e b oun d with the previous r esults. • Herbster et. al. [6, 5] pr esen t an algorithm based on p erceptron and pro ve the b ound of 4 · | cut ( ℓ ) | · R G + 2 , for the n u m b er of mistak es where R G is the large st effectiv e r esistance b etw een an y pair of no d es in G (see [5], for the formal definition). W e note that th er e are graphs where R G is la rge, e.g , for line graph R G = n − 1. Ou r b ound is b etter when R G = Ω(log n ). While in the w orst case R G can b e large, for man y classes of graphs , e.g., highly connected graphs w ith small diameter, R G can b e v ery small. In [5], th ey giv e an example where the cut size | cut ( ℓ ) | is linear, while R G is O (1 / | cut ( ℓ ) | ). In this examp le, their m istak e b ound remains constant, while our b ound gro ws with | cut ( ℓ ) | . • In a recent pap er, Herbster [4] exploits the cluster structures of graphs and p ro v es the b ound of N ( G , ρ ) + 4 · | cut ( ℓ ) | · ρ + 1 for an y ρ > 0, on the num b er of mistakes, wher e N ( X, ρ ), the co vering num b er, is the minim um n u m b er of sets of diameter ρ that con tain all v ertices of G under the semi-norm induced b y the graph L aplacian (see [4] for definitions). This b ound impro v es o v er pr evious b ound in [5] wh en the graph has s mall n u mb er of clusters with s m all diamete rs. Herbs ter giv es an example where the new algo rithm m ak es only a constant n umb er of mistake s wh ile the algorithm fr om [5] mak es linear mistak es. Again, in this example, our algorithm has linear mista ke b ound. W e n ote that there is a trade-off b et w een the diameter ρ of clusters and the num b er clusters in Herbster’s b ound. F or man y classes of graphs with large d iameter, e.g. line graphs, using cluster stru ctur e d o es not h elp. The dep endent on th e cut size can s till be Ω( n ) for graphs with n v ertices. • P elc km ans and Suykens [7] present a simple combinatorial algorithm and sh o w that th e set M of v ertices wh er e the algo rithm pr edicts in corr ectly satisfies P v ∈ M d M ,v ≤ 4 · | cut ( ℓ ) | , where d M ,v is th e num b er of v ertices adjacen t to v that is also in M . Note that their b ound only account s for edges b et w een t wo mistak en v ertices. If there are n o edges b et we en v ertices in M , their b ound do es not sa y an ythin g. F or example, consider the ca se with line graph with n vertice s, wh ere v ertices 1 , 2 , . . . , n/ 2 h a v e lab el +1 and v ertices n/ 2 + 1 , . . . , n ha v e lab el − 1. The algorithm of P elc k m ans and Suykens can make Ω( n ) mistak es if an adve rsary asks the labels of 1 , 3 , 5 , . . . , while the cut size is just 1. 4 Op en questions and discu ssions Our b ound d ep ends on the worst case b oun d on the congestion from th e routing pr oblem. Ho wev er, the O (log n ) b oun d seems v ery loose for dense graphs. It wo uld b e nice to see if one can fi nd the co nnection b et we en the w orst case congestion a nd the effectiv e resistance. W e note that when th e effec tiv e resistance is lo w, b et ween any t wo no des there must b e man y short disjoin t paths, and this should help redu cing the co ngestion. Also, there is extensive literature on online routing with small congestion (see, e.g., [8 , 3, 9]). Can these results b e used to giv e b etter mistake b ounds as w ell? W e note that our pro of cannot giv e a mistak e b ound smaller than | cut ( ℓ ) | . T o imp ro v e further, one need a w ay to ac count for cut edges that ha v e not b een c harged. Finally , w e wish to see an y adversarial b ound on the n u mb er of mistak es f or an online lab el prediction algorithm. In this pap er , we h av e shown that our algorithm is optimal (up to a constan t f actor) for line graph s. Th e ultimate goal w ould b e to fin d an optimal algorithm for general graphs . References [1] Baruch Aw erbu c h and Y ossi Azar. Comp etitiv e multic ast routing. Wir el. Netw. , 1(1):1 07– 114, 1995. [2] Parin ya Ch alermso ok a nd Jittat F ak c haro enph ol. Simple distributed algorithms for appro x- imating minim u m steiner trees. In COCOON , p ages 380–389, 2005 . [3] C hris Harrelson, Kir sten Hildrum, and Satish Rao. A p olynomial-time tree decomp osition to min im ize co ngestion. In SP A A ’03: Pr o c e e dings of the fifte enth annual ACM symp osium on Par al lel algorithms and ar chite ctur e s , pages 34–43, New Y ork, NY, USA, 20 03. A CM. [4] Mark Herbster. Exploiting cluster-structure to pr edict the lab eling of a graph. In AL T’08 (T o app e ar) , 2008. [5] Mark Herbster and Massimiliano Po ntil . Prediction on a graph w ith a p erceptron. In B. Sc h¨ olk opf , J. Pla tt, and T. Hoffman, editors, A dvanc es in Neur al Information Pr o c e ssing Systems 19 , p ages 577–584. MIT Press, Cam br id ge, MA, 200 7. [6] Mark Herbster, Massimiliano Po ntil, and Lisa W ainer. Onlin e learning o ver graphs. In ICML ’05: Pr o c e e dings of the 22nd i nternational c onfer enc e on Machine le arning , pages 305–3 12, New Y ork, NY, USA, 2005. A CM. [7] K ristiaan P elc kmans and Johan A.K. S uyk ens. An online algorithm for learning a lab eling of a graph . In Pr o c e e dings of the 6th Interna tional W orkshop on M ining and L e arning with Gr aphs , 2008 . [8] Harald R¨ a c k e. Minimizing co ngestion in general netw orks. In FOCS ’02: Pr o c e e dings of the 43r d Symp osium on F oundations of Computer Scienc e , p ages 43–52, W ashington, DC, USA, 20 02. IEEE Compu ter So ciet y . [9] Harald R¨ a c k e. Optimal h ierarc hical decomp ositions for congestion min imization in net works. In STOC ’08: Pr o c e e dings of the 40th a nnual ACM symp osium on Th e ory of c omputing , pages 255–264, New Y ork, NY, USA, 20 08. A CM.