Minimum Entropy Orientations

Minim um En trop y Orien tations Jean Cardinal ∗ Sam uel Fiorini † Gw ena¨ el Joret ‡ Abstract W e study graph orientations that minimize the en tropy of the in-degree sequence. W e pro ve that the minimum entrop y orientation problem is NP-hard even if the graph is planar, and that there exists a simple linear-time algorithm that returns an approximate solution with an additive error guaran tee of 1 bit. Keyw ords: Appro ximation algorithm; En tropy; Graph orientation; V ertex cov er 1 In tro duction All graphs considered here are finite, undirected and lo opless, but multiple edges are allow ed. Let G = ( V , E ) b e a graph with n vertices and m edges, and consider an y directed graph ~ G obtained by orienting the edges of G . The in-de gr e e distribution p ∈ Q V + of this orientation is defined by p v := ρ ~ G ( v ) /m , where ρ ~ G ( v ) denotes the in-degree of v in ~ G . In this pap er, we consider the problem of finding an orientation whose corresp onding in-degree distribution is as un balanced as p ossible. As a balance measure, we use the entr opy H ( p ) := X v ∈ V − p v log p v , where log denotes the base 2 logarithm, and − 0 log 0 := 0. The minimum entr opy orientation problem (MINEO) is the problem of finding an orien tation of G with an in-degree distribution p minimizing H ( p ). The study of MINEO is motiv ated by that of the minimum entr opy set c over problem (MINESC), introduced b y Halp erin and Karp [8]. In the latter problem, w e are given a ground set U and a collection S = { S 1 , . . . , S q } of subsets of U whose union is U , and we ha ve to assign eac h elemen t of U to a subset S i con taining it. This assignment partitions U into classes U 1 , U 2 , . . . , U q of elements assigned to the same subset, and the ob jectiv e is to minimize the entrop y of the probabilit y distribution | U i | | U | defined by this partition. Hence, MINEO is the sp ecial case of MINESC where eac h element of the ground set can b e co vered b y exactly tw o sets from S . (T o see this, take U := E and let S := { S v | v ∈ V } with S v := { e ∈ E | e is inciden t to v } .) ∗ Univ ersit´ e Libre de Bruxelles, D ´ epartement d’Informatique, c.p. 212, B-1050 Brussels, Belgium, jcardin@ulb.ac.b e. † Univ ersit´ e Libre de Bruxelles, D´ epartemen t de Math´ ematique, c.p. 216, B-1050 Brussels, Belgium, sfior- ini@ulb.ac.b e. ‡ Univ ersit´ e Libre de Bruxelles, D ´ epartement d’Informatique, c.p. 212, B-1050 Brussels, Belgium, g joret@ulb.ac.be. G. Joret is a Research F ellow of the F onds National de la Recherc he Scientifique (F.R.S.– FNRS). 1 Figure 1: Construction of tigh t examples for the greedy algorithm. Giv en a v alue t (in our illustration, t = 4), let S b e a set of t ! indep enden t vertices (the red vertices). Then for each i in { 1 , 2 , . . . , t } , construct t ! /i independent vertices of degree i with neighbors in S , suc h that their neighborho o ds partition S into subsets of size i . The optimal solution is obtained by orien ting eac h edge to ward its endp oint in S . The greedy algorithm ma y orient each edge in the opp osite direction. This example is equiv alen t to the tight example previously given for the minimum en tropy set cov er problem [3]. The main motiv ation of Halp erin and Karp for introducing MINESC was to solv e a hap- lot yping problem, of imp ortance in computational biology . This problem in volv es co vering a set U of p artial haplotyp es of length d , defined as words in the set { 0 , 1 , ∗} d , by c omplete haplotyp es , defined as w ords in { 0 , 1 } d . Eac h subset of S corresp onds to a complete hap- lot yp e h ∈ { 0 , 1 } d and con tains all partial haplotypes that are c omp atible with h , that is, partial haplot yp es whose sym b ols match in ev ery non-‘ ∗ ’ p ositions with h . The ‘ ∗ ’ p ositions in the partial haplotypes are interpreted as measuremen ts error. It is shown that, under some probabilistic assumptions, minimizing the entrop y of the co vering amounts to maximizing its lik eliho o d [8]. An application of MINEO is the sp ecial case of partial haplot yping in whic h eac h partial haplot yp e has at most one ‘ ∗ ’ in it. P artial haplotypes are then edges of a d -dimensional h yp ercub e, and the subsets S i corresp ond to vertices of this hypercub e. In other words, this sp ecial case of the partial haplotyping problem is a minimum entrop y orientation problem in a partial h yp ercube. The well-kno wn greedy algorithm for the set co ver problem is applicable to MINEO. It inv olv es iterating the following steps: choose a maximum degree vertex v in G , orient all edges inciden t to v tow ard v , and remov e v from G . The p erformance of the greedy algorithm for MINESC has b een studied thoroughly . Halperin and Karp [8] first show ed that the greedy algorithm approximates MINESC to within some additiv e constant. Then the current authors impro ved their analysis and sho wed that the greedy algorithm returns a solution whose en tropy is at most the optimum plus log e bits, with log e ≈ 1 . 4427 bits [3]. Moreo ver, they prov ed that it is NP-hard to appro ximate MINESC to within an additiv e error of log e −  , for every constan t  > 0. Since MINEO is a sp ecial case of MINESC, the first result implies that the greedy algorithm also approximates MINEO to within an additive error of log e bits. W e note that there exist instances of MINEO where the latter b ound is (asymptotically) attained; see for example those describ ed in Figure 1. In this pap er, we first prov e that MINEO is NP-hard, ev en if the input graph is planar (Section 2). The reduction is from a restricted v ersion of the 1-in-3 Satisfiability problem. Then, we show in Section 3 that there exists a simple linear-time approximation algorithm for MINEO with an improv ed approximation guarantee of 1 bit. T o conclude the in tro duction, w e men tion that MINEO is also related to the minimum sum vertex c over problem (MINSVC), in tro duced by F eige, Lo v´ asz, and T etali [5]. In that 2 problem, w e are given a graph G , and ha ve to find an ordering v 1 , v 2 , . . . , v n of its vertices suc h that the av erage cov er time of an edge is minimized, that is, minimizing n X i =1 i · | f ( v i ) | , where f ( v i ) denotes the set of edges that are incident to v i but to no vertex v j with j < i . This problem can also be seen as a graph orien tation problem in whic h the most unbalanced orien tation is sough t, although with a differen t balance measure than that of MINEO. F eige et al. [5] prov ed that MINSVC is APX-hard, and gav e a 2-approximation algorithm, based on randomized rounding of a natural linear programming relaxation. Using a differen t rounding tec hnique, Berenholz, F eige, and Peleg [1] recently derived an improv ed approximation factor of 1 . 99995. Although MINEO and MINSVC share some common properties, the main difference b e- t ween these tw o problems is p erhaps ho w they b eha ve with respect to instances that are the union of smaller ones: As noted by Berenholz et al. [1], MINSVC is not “linear”, in the sense that an optimal solution to the union of tw o disjoint graphs G 1 and G 2 is not necessarily a com bination of an optimal solution to G 1 and G 2 , resp ectiv ely . (In particular, the APX- hardness proof of F eige et al. [5] relies on this non-linearity .) On the other hand, MINEO is linear, as can b e easily c hec ked. 2 Hardness Let a = ( a 1 , a 2 , . . . , a n ) and b = ( b 1 , b 2 , . . . , b n ) b e tw o sequences of non-negative integers sorted in non-increasing order, and such that P n i =1 a i = P n i =1 b i =: m . W e say that sequence a dominates sequence b if i X j =1 a j ≥ i X j =1 b j (1) for ev ery i ∈ { 1 , . . . , n } , and moreov er (1) holds with strict inequality for at least one such i . W e emphasize that a 6 = b whenev er a dominates b . The follo wing lemma is a standard consequence of the strict concavit y of the function x 7→ − x log x ; see e.g. [2, 6, 9] for different pro ofs. Lemma 1. If a dominates b , then H ( a/m ) < H ( b/m ) . Theorem 1. Finding a minimum entr opy orientation of a planar gr aph is NP-har d. Pr o of. In the 1-in-3 Satisfiability problem, we are giv en a 3-SA T formula in input, and we ha ve to decide whether there exists a truth assignment of the v ariables suc h that eac h clause is satisfied b y exactly one of its three literals. Mo ore and Robson [12] pro v ed that this problem is NP-complete, even if ev ery v ariable app ears in exactly three clauses, there is no negation in the form ula, and the bipartite graph obtained by linking a v ariable and a clause if and only if the v ariable appears in the clause, is planar. W e will reduce the latter restriction of the 1-in-3 Satisfiabilit y problem to MINEO. It will b e conv enien t for the pro of to restate Mo ore and Robson’s result in the context of the Exact Co ver problem. The latter asks, given a set system ( U, S ), to decide if U can b e cov ered using pairwise disjoin t sets from S . As noted by Li and T oulouse [11], the NP-completeness 3 u j, 1 u j, 2 u j, 3 v j, 3 v j, 2 v j, 1 Figure 2: Gadget for elemen t u j . of the version of 1-in-3 Satisfiability describ ed ab o ve directly implies that Exact Cov er is NP- complete even when ev ery set in S has cardinality exactly 3, each element in U is included in exactly three sets of S , and the “elemen ts v ersus sets” incidence graph is planar. (T o see this, consider the set system ( U, S ) where U and S are asso ciated with the clauses and the v ariables of the 1-in-3 Satisfiability instance, resp ectiv ely .) Let ( U, S ) b e an y such set system, and denote by u 1 , . . . , u q and S 1 , . . . , S q the elements of U and the sets in S , resp ectively . W e ma y assume without loss of generalit y that q is a multiple of 3, since otherwise ( U, S ) has no exact cov er. W e construct a graph G = ( V , E ) as follows: First, create a vertex s i p er set S i . Then, for eac h elemen t u j , add a cop y of the gadget depicted in Figure 2, and link u j,k (1 ≤ k ≤ 3) to S j k , where j 1 , j 2 , j 3 are the indices of the three sets containing u j . The fact that G is planar directly follows from the planarit y of the bipartite graph underlying the set system ( U, S ). Let ~ G b e any orientation of G with minimum entrop y , and denote by A its arc set. F or a subset X ⊆ V of vertices, w e use δ ( X ) for the n umber of arcs going from X to V − X in ~ G . W e define the in-de gr e e se quenc e of X , denoted in-seq( X ), as the sequence of in-degrees of the v ertices in X , sorted in non-increasing order. Let also X j := { u j, 1 , u j, 2 , u j, 3 , v j, 1 , v j, 2 , v j, 3 } , for every j ∈ { 1 , . . . , q } . Claim 1. The in-de gr e e se quenc e of the set X j in ~ G is given by the fol lowing table: δ ( X j ) in-se q ( X j ) 0 (4 , 3 , 3 , 1 , 1 , 0) 1 (4 , 3 , 3 , 1 , 0 , 0) 2 (4 , 3 , 2 , 1 , 0 , 0) 3 (3 , 3 , 2 , 1 , 0 , 0) Pr o of. Denote by ( c 1 , c 2 , . . . , c 6 ) the in-degree sequence of the set X j in ~ G . Arguing by con tradiction, w e supp ose that this sequence is different from the one given in the table. Considering the structure of the elemen t-gadget, and in particular its t wo disjoint triangles, w e infer the following b ounds: c 1 ≤  4 if δ ( X j ) < 3; 3 otherwise; c 2 ≤ 3; c 3 ≤ 3; c 4 ≥ 1; c 5 ≥ 1 if δ ( X j ) = 0 . 4 Figure 3: A go o d orien tation of the element-gadget. It is assumed that the arcs going out of X j o ccur clo c kwise from the top of the figure. It follows from the inequalities ab o v e and P 6 ` =1 c ` = 12 − δ ( X j ) that in-seq( X j ) is dominated (in the sense of Lemma 1) b y the corresp onding sequence in the table. Now, it is alwa ys p ossible to re-orien t the arcs of ~ G with b oth endpoints in X j in suc h a wa y that in-seq( X j ) realizes the latter sequence, as illustrated in Figure 3. Because this mo dification leav es the in-degrees of the vertices in V − X j unc hanged, w e deduce from Lemma 1 that the new orien tation has an entrop y strictly smaller than ~ G , a con tradiction. Using Claim 1, w e may assume without loss of generality that ~ G [ X j ] is isomorphic to the orien tation of the element-gadget given in Figure 3. Renaming the vertices if necessary , w e ma y th us supp ose that u j, 1 , u j, 2 , u j, 3 ha ve in-degree 0, 2, 3 in ~ G [ X j ], resp ectiv ely . Consider now the three edges u j, 1 s j 1 , u j, 2 s j 2 , u j, 3 s j 3 of G . Recall that j 1 , j 2 , j 3 denote the indices of the three sets in S containing the element u j . Because ~ G has minimum entrop y , w e ma y assume that the first of these three edges is oriented out of X j and the last tw o to ward X j . Indeed, if we hav e ( u j, 3 , s j 3 ) ∈ A then ρ ~ G ( s j 3 ) ≤ 3, ρ ~ G ( u j, 3 ) = 3, and changing the orientation of u j, 3 s j 3 w ould decrease the entrop y of ~ G by Lemma 1, a contradiction. Moreo ver, if ( u j, 2 , s j 2 ) ∈ A , then re-orienting u j, 2 s j 2 either leav es the entrop y of ~ G unc hanged or decreases it. A similar argumen t holds when ( s j 1 , u j, 1 ) ∈ A . It follows that there is exactly one arc going out of X j in ~ G , for ev ery j ∈ { 1 , . . . , q } . W e pro ceed with a second (and last) claim. Claim 2. L et m := | A | ( = 12 q ). Then the entr opy of ~ G is at le ast 1 m (4 q log ( m/ 4) + 7 q log ( m/ 3) + q log m ) , with e quality if and only if ther e exists an exact c over of ( U, S ) . Pr o of. As w e hav e seen, we hav e without loss of generality in-seq( X j ) = (4 , 3 , 3 , 1 , 0 , 0) for ev ery j ∈ { 1 , . . . , q } . Then, eac h comp onen t of in-seq( S ), where S := { s 1 , s 2 , . . . , s q } , is clearly at most 3, and the sum of all of them equals q . Combining these observ ations with Lemma 1, we deduce that the entrop y of ~ G is at least the low er b ound given in the claim, and that equality o ccurs if and only if in-seq( S ) equals (3 , 3 , . . . , 3 , 0 , 0 , . . . , 0). W e show that the latter happens if and only if there exists an exact cov er of ( U, S ). Supp ose first in-seq( S ) = (3 , 3 , . . . , 3 , 0 , 0 , . . . , 0), and define S ∗ ⊆ S as S ∗ := { S i : ρ ~ G ( s i ) > 0 } . 5 It is then easily seen that the collection S ∗ is an exact cov er for the set system ( U, S ). No w assume that S 0 ⊆ S is an exact cov er of ( U, S ). After p erm uting the indices j 1 , j 2 and j 3 , we can assume that the set of S 0 con taining u j is S j 1 . Orien ting each edge s i u j,k of G tow ard s i if k = 1, tow ard u j,k otherwise, and using the orientation of the element-gadgets giv en in Figure 3, we obtain an orientation ~ G ∗ of G where eac h X j has in-degree sequence (4 , 3 , 3 , 1 , 0 , 0) and S has in-degree sequence (3 , 3 , . . . , 3 , 0 , 0 , . . . , 0). Hence, S must also hav e the same in-degree sequence in ~ G , since otherwise ~ G ∗ w ould hav e entrop y strictly less than ~ G , contradicting the optimality of the latter orientation. The claim follows. By Claim 2, a p olynomial-time algorithm finding a minimum entrop y orientation of G could b e used to decide, in p olynomial time, if there exists an exact co ver of ( U, S ). This completes the proof of the theorem. Let us say that a graph orientation problem has the strict dominanc e pr op erty if the ob jective function F to minimize is such that F ( ~ G ) < F ( ~ G 0 ) whenev er ~ G and ~ G 0 are t wo orientations of a graph G = ( V , E ) suc h that the in-degree sequence of V in ~ G dominates that of V in ~ G 0 . Hence, Lemma 1 says exactly that MINEO has the strict dominance prop ert y . W e remark that, since the pro of of Theorem 1 relies solely on that lemma, it follows more generally that ev ery orientation problem with the strict dominance prop ert y is NP-hard on planar graphs. 3 Appro ximation Throughout this section, we denote by OPT( G ) the minim um entrop y of an orien tation of G . An orien tation of G is biase d if eac h edge v w with deg ( v ) > deg ( w ) is oriented tow ard v . It turns out that biased orientations hav e entrop y close to the minimum ac hiev able: Theorem 2. The entr opy of any biase d orientation of G is at most OPT ( G ) + 1 . Since finding a biased orientation can easily b e done in linear time, Theorem 2 yields the follo wing corollary: Corollary 1. MINEO c an b e appr oximate d within an additive err or of 1 bit, in line ar time. Let m denote the num b er of edges of G and d v := deg( v ) / (2 m ) the normalized degree of a v ertex v . Given tw o discrete probabilit y distributions p and q o ver a common domain X , w e denote by D ( p k q ) their relativ e entrop y (or Kullbac k-Leibler distance), defined as follo ws: D ( p k q ) := X i ∈ X p i log p i q i . It is known that D ( p k q ) is alw ays non-negative (see for instance Cov er and Thomas [4, Section 2.6] for a pro of ). 6 Pr o of of The or em 2. Let ~ G be an optimal orien tation of G . Denote respectively b y p and A the in-degree distribution and arc set of ~ G . W e first rewrite the en trop y of p as follo ws: OPT( G ) = X v ∈ V − p v · log p v = X v ∈ V − ρ ~ G ( v ) m · log ρ ~ G ( v ) m = log m − 1 m X v ∈ V ρ ~ G ( v ) · log ρ ~ G ( v ) = log m − 1 m X ( u,v ) ∈ A log ρ ~ G ( v ) . No w w e observ e that for an y v ertex v we hav e ρ ~ G ( v ) ≤ deg( v ). W e let ~ G [ b e a biased orien- tation, A [ its arc set, and p [ the corresp onding in-degree distribution. F rom our observ ation, w e ha v e OPT( G ) ≥ log m − 1 m X ( u,v ) ∈ A log max { deg( u ) , deg( v ) } = log m − 1 m X ( u,v ) ∈ A [ log deg( v ) (b ecause ~ G [ is biased) = log m − 1 m X v ∈ V ρ ~ G [ ( v ) · log deg( v ) = X v ∈ V − p [ v · log deg( v ) m = X v ∈ V − p [ v · log d v ! − 1 = X v ∈ V − p [ v · log p [ v ! + X v ∈ V p [ v · log p [ v d v ! − 1 = H ( p [ ) + D ( p [ k d ) − 1 Since D ( p [ k d ) ≥ 0, w e hav e H ( p [ ) ≤ OPT( G ) + 1, which concludes the pro of. W e note that the b ound given in Theorem 2 is tight: consider for instance the case where G is a cycle. W e end this section with some remarks. F or S ⊆ V , let e ( S ) denote the fraction of edges of G inciden t to a vertex in S . Thus e ( V ) = 1. It is well-kno wn that e is a submo dular function, that is, satisfies e ( X ) + e ( Y ) ≥ e ( X ∩ Y ) + e ( X ∪ Y ) for all X, Y ⊆ V . W e denote the base polytop e of e by P ( G ). Letting p ( S ) := P v ∈ S p v for S ⊆ V , w e th us hav e P ( G ) = { p ∈ R V : p ( S ) ≤ e ( S ) ∀ S ⊆ V , p ( V ) = 1 } . 7 It follows from standard results on p olymatroids that P ( G ) is the conv ex hull of all the in- degree distributions of orientations of G ; see for instance Schrijv er [14]. The vertices of P ( G ) corresp ond to the acyclic orientations of G . Consider now the following generic linear program, where for each v ∈ V , c v is a fixed non-negativ e cost: min X v ∈ V c v · p v s.t. p ∈ P ( G ) . (2) This linear program can b e solved by the follo wing greedy algorithm: First order the vertices in V in non-decreasing order of costs, say v 1 , v 2 , . . . , v n . Then, start with the null vector p := (0 , 0 , . . . , 0), and, for eac h i = 1 , . . . , n , increase the i th comp onen t of p as muc h as p ossible, ensuring that p ( S ) ≤ e ( S ) remains true at all time, for every S ⊆ V . It is well- kno wn (see e.g. [14]) that the resulting p oint p b elongs to P ( G ), that is, it satisfies also p ( V ) = 1, and furthermore that p is an optimal solution to the ab o ve linear program. Let us set c v := − log (deg( v ) /m ). Thus, w e obtain the following linear program: min X v ∈ V − p v · log deg( v ) m s.t. p ∈ P ( G ) . (3) Since MINEO can b e formulated as min X v ∈ V − p v · log p v s.t. p ∈ P ( G ) , (4) and − p v · log p v ≥ − p v · log (deg( v ) /m ) trivially holds for every p ∈ P ( G ), the optimum v alue of (3) gives a lo wer b ound on OPT( G ). No w, observ e that performing the greedy algorithm to solv e (3) amounts to finding a biased orientation of G . Moreo ver, the in-degree sequence of every suc h orien tation can b e pro duced b y the algorithm. T o conclude, w e mention that the natural counterpart of MINEO where one aims at finding an orien tation of G with maximum en trop y is p olynomial. This is b ecause maximizing a separable conca v e function o v er P ( G ) ∩ 1 m Z V can be done in polynomial time; see [7, 10, 13]. Ac kno wledgmen ts The authors wish to thank Olivier Roussel from the ´ Ecole Normale Sup ´ erieure de Cac han for his preliminary inv estigations on this topic. W e also thank an anonymous referee for an observ ation that made the pro of of Theorem 2 more concise. This work was supp orted by the A ctions de R e cher che Conc ert´ ees (AR C) fund of the Communaut ´ e fr an¸ caise de Belgique . References [1] U. Berenholz, U. F eige, and D. P eleg. Impro v ed appro ximation for min-sum v ertex co v er. T echnical Rep ort MCS06-07, Computer Science and Applied Mathematics, W eizmann Institute of Science, 2006. 8 [2] J. Cardinal, S. Fiorini, and G. Joret. Minim um entrop y coloring. In Pr o c e e dings of the 16th International Symp osium on Algorithms and Computation (ISAAC 2005) , volume 3827 of L e ctur e Notes in Computer Scienc e , pages 819–828, Berlin, 2005. Springer. [3] J. Cardinal, S. Fiorini, and G. Joret. Tigh t results on minim um entrop y set cov er. A lgorithmic a , 51(1):49–60, 2008. [4] T. M. Cov er and J. A. Thomas. Elements of Information The ory . Wiley-Interscience, second edition, 2006. [5] U. F eige, L. Lov´ asz, and P . T etali. Approximating min sum set co ver. Algorithmic a , 40(4):219–234, 2004. [6] T. F ukunaga, M. M. Halld´ orsson, and H. Nagamo c hi. Robust cost colorings. In SOD A ’08: Pr o c e e dings of the ninete enth annual ACM-SIAM symp osium on Discr ete algorithms , pages 1204–1212, Philadelphia, P A, USA, 2008. Society for Industrial and Applied Math- ematics. [7] H. Gro enevelt. Tw o algorithms for maximizing a separable concav e function ov er a p olymatroid feasible region. Eur op e an Journal of Op er ational R ese ar ch , 54(2):227–236, 1991. [8] E. Halp erin and R. M. Karp. The minim um-entrop y set co ver problem. The or et. Comput. Sci. , 348(2-3):240–250, 2005. [9] G. H. Hardy , J. E. Littlew o o d, and G. P´ oly a. Ine qualities . Cam bridge Mathematical Library . Cam bridge Univ ersit y Press, Cam bridge, 1988. Reprint of the 1952 edition. [10] D. S. Hoch baum. Low er and upp er bounds for the allo cation problem and other nonlinear optimization problems. Math. Op er. R es. , 19(2):390–409, 1994. [11] P . C. Li and M. T oulouse. V ariations of the maximum leaf spanning tree problem for bipartite graphs. Inform. Pr o c ess. L ett. , 97(4):129–132, 2006. [12] C. Moore and J. M. Robson. Hard tiling problems with simple tiles. Discr ete Comput. Ge om. , 26(4):573–590, 2001. [13] S. Moriguchi and A. Shioura. On Hoch baum’s pro ximity-scaling algorithm for the general resource allo cation problem. Math. Op er. R es. , 29(2):394–397, 2004. [14] A. Schrijv er. Combinatorial optimization. Polyhe dr a and efficiency. Vol. B , volume 24 of Algorithms and Combinatorics . Springer-V erlag, Berlin, 2003. Matroids, trees, stable sets, Chapters 39–69. 9