Submodular Function Maximization via the Multilinear Relaxation and Contention Resolution Schemes

Submodul ar Function Maximization via the Multilin ear Relaxation and Contention Resolution Schemes ∗ Chandra Chekuri † Jan V ondr ´ ak ‡ Rico Zenklusen § August 14, 2014 Abstract W e co nsider the problem of ma ximizing a non-ne gati ve submo dular set function f : 2 N → R + ov er a groun d set N subject to a v ariety of packing type constraints includ ing (multiple) matroid constraints, knap sack constraints, and their intersections. In this paper we de velop a general framework that allows us to derive a number of new results, in particular when f may be a non-mon otone function. Our alg orithms are based on (approximately) maximizing the multilinear extension F of f [6] over a polytope P that represents t he constraints, and then effecti vely rounding the fractional solution. Although this approach has been used quite successfully [7, 33 , 36, 15, 4], it has been limi ted in some important way s. W e o v ercome these limitati ons as follo ws. First, we gi ve c onstant f actor approx imation algorithms to maximize F over a do wn-closed polytope P described by an efﬁcient separation oracle. P rev iously this was k no wn only for monotone f unctions [49]. For non-monoton e functions, a constant factor was known only when the polytope w as either t he intersection of a ﬁ xed number of knapsack constraints [36] or a matroid polytope [50, 43]. Second, we show that co ntention r esolution sc hemes are an effectiv e way t o round a fraction al solution, ev en when f is non-mono tone. In parti cular , con tention resolution schemes for different polytopes can be combined to handle the intersection of different constraints. V ia LP duality we show that a conten tion resolution scheme f or a co nstraint is related to the corr elation gap [2] of weighted rank functions of the constraint. This leads to an optimal contention resolution scheme for the matroid polytope. Our results pro vide a broadly applicable frame work for maximizing l inear and submod ular functions subject to ind ependence constraints. W e gi ve several illustrative examples. Contention reso lution schemes may ﬁnd other applications. 1 Introd uction W e consider the meta-prob lem o f maximizing a non-n egativ e subm odular set function subject to independen ce co n- straints. Formally , let N be a ﬁnite gro und set of cardinality n , and let f : 2 N → R + be a sub modular set fu nction over N . 1 Let I ⊆ 2 N be a downward-closed family 2 of subsets of N . Our problem is then max S ∈I f ( S ) . W e are interested in indepen dence families indu ced by natural and useful constraints such as matroid constraints, kn apsack constraints, related special cases, and the ir intersections. Th rough out th is pape r we assume that f is g iv en via a value oracle; that is, gi ven a set S ⊆ N the oracle returns f ( S ) . The fu nction f cou ld be m onoto ne or non- monoto ne 3 ; monoto ne functions typically allo w better ap proxim ation resu lts. Submod ular fu nction m aximization has r ecently attracted con siderable atten tion in theo retical c omputer science. This is fo r a variety of reasons, includin g di verse applications—a pro minent application ﬁeld being algorith mic game ∗ A preli minary version of this pap er appeared in Proc. o f ACM ST OC , 2011. † Dept. of Compute r Scie nce, Univ . of Illinoi s, Urbana, IL 61801, USA. Partial ly supported by NSF grants CCF-072878 2 and CCF-1016684. E-mail: chekuri@illinois.edu . ‡ IBM Almaden Researc h Center , San Jose, CA 95120, USA. E-mail: jvondrak@us.ibm.co m . § Dept. of Mathema tics, E TH Zurich, 8092 Zurich, Switzerl and, and Department of Applied Mathemati cs and Statistics, Johns Hopkins Univ er- sity , Baltimore , MD 21218, USA. E-mail: rico z@math.ethz.ch . Supported by S wiss National Science Foundat ion grant PBEZP2-129524, by NSF grants CCF-111584 9 and CCF-0829878, and by ONR grants N00014-12-1-00 33, N00014-11-1-005 3 and N00014-09-1-0326. 1 A set functio n f : 2 N → R is submodular iff f ( A ) + f ( B ) ≥ f ( A ∪ B ) + f ( A ∩ B ) for all A, B ⊆ N . 2 A fa mily of sets I ⊆ 2 N is do wnward-c losed if for any A ⊂ B ⊆ N , B ∈ I implies that A ∈ I . 3 f is monotone if f ( A ) ≤ f ( B ) whenev er A ⊆ B . 1 theory , where submodular fun ctions are very commo nly used as utility functions t o describe dimin ishing retu rns—and also the recognition of interesting algorithmic and structura l properties. A numb er of well-k nown p roblems can be seen as special cases of submodu lar function max imization. For example, the APX-hard Ma x-Cut pr oblem ca n b e seen as (unconstrained ) maximization o f the cut functio n f : 2 V → R + of a graph G = ( V , E ) . (Note that f here is non-m onoton e.) Another well-known specia l case of o ur problem is the Max - k -Cover problem, which can be viewed as max { f ( S ) : | S | ≤ k } where f ( S ) = | S j ∈ S A j | is the coverage fun ction for a collec tion of sets { A i } . Max - k - Cover is ha rd to approximate t o within a factor of (1 − 1 /e + ε ) for any ﬁ xed ε > 0 , unless P = N P [20]. Hence we focus on approx imation alg orithms 4 . Classical w ork in submodular function maximization was based on combinato rial tech niques such as the gre edy algorithm and lo cal search. W e mention the work of Cornuejols, Fisher , Nemhau ser and W olse y [18, 4 2, 25, 41] from th e late 70’ s which showed a variety of app roximatio n bounds when f is monoton e submodu lar and I is the intersection of matroid constrain ts. Recent algo rithmic work ha s considerab ly extended and impr oved the classical results. Local-search methods hav e been id entiﬁed as particu larly useful, espec ially for n on-mo noton e f unctions. Some of the recent re sults include the ﬁrst c onstant f actor ap proxim ation for the u nconstrain ed submod ular fu nction maximization problem [2 1], and a variety of approxim ation results for k napsack and m atroid c onstraints [36, 37]. Th e greedy algorithm has also been modiﬁed and made applicable to non-mo noton e function s [29]. Despite the above-men tioned re sults, combinator ial techniqu es have some limitations: ( i) they have not been able to achieve optim al approximation results, e xcept in the basic case of a single card inality or knapsack c onstraint [42, 46]; (ii) they do not provide the ﬂexibility to combine constraints of d ifferent types. A ne w appr oach which overcomes some of these obstacles and brin gs submodu lar f unction maximizatio n closer t o the world of polyh edral techniques is via the multilinear r e laxation , introd uced in this context in [6]. A relaxation and rounding framework based on the multilinear relaxation. In this paper we introduc e a gen- eral relaxation and ro undin g f ramework for maximizin g su bmodu lar f unction s, which builds upon, and sig niﬁcantly extends, p revious approache s. When dealing with linear (or co n vex) o bjective f unctions, a standa rd paradig m is to design a linear or conve x relaxation wh ose solution is then round ed via a problem -speciﬁc proced ure. A difﬁculty faced in extending this ap proach to maximizin g a submo dular fun ction f : 2 N → R + — which we often interpret as a function on the vertices o f a { 0 , 1 } N hyperc ube tha t correspo nd to incidence vectors — is to ﬁnd a su itable extension g : [0 , 1] N → R + of f to the full hyperc ube. The goal is to leverage such an extension g as follows. Supp ose we have a polytop e P I ⊆ [0 , 1] N that is a relaxatio n for I ⊆ 2 N in th e sense that { 1 I | I ∈ I } ⊂ P I . W e w ant to approx imately ma ximize the con tinuous p roblem max x ∈ P I g ( x ) to ﬁnd a fraction al solutio n x ∗ ∈ P I that is ﬁna lly round ed to a fea sible integral s olution. The best-stud ied extension of a sub modula r function is the L ov ´ asz extension [39]; howev er , being a conve x func- tion, it is m ostly suitable for submodular fun ction minimiz ation prob lems. For maximization o f submodu lar functions, the following multilinear e xtension w as introdu ced in [6], inspired by the w ork in [1]: F ( x ) = X S ⊆ N f ( S ) Y i ∈ S x i Y j 6∈ S (1 − x j ) . The v alue F ( x ) is equi valently the e xpected v alue of f ( R ) where R is a random set o btained by picking each element i independ ently with pr obability x i . W e observe that if f is modu lar 5 then F is simply a line ar function . In this paper we fo cus on the m ultilinear extension. The two obvio us questions that ar ise when try ing to build a gen eral relaxation and rounding framew ork ba sed on the multilinear extensio n are the follo wing. First, can we (approximately ) solve t he problem max x ∈ P I F ( x ) ? This question is particularly in teresting due to th e fact that the multilinear extension is in general not concave (n or con vex). Second , can we round a fractional solution ef fectiv ely? Recent work has ad dressed the above que stions in se veral ways. First, V ondr ´ ak [ 49] ga ve a continuou s greedy algorithm that gi ves an optimal (1 − 1 /e ) -ap prox imation fo r the pro blem max x ∈ P F ( x ) when f is monoton e sub- modular an d P is a solv able polytop e 6 . When f is non -mono tone, the picture is less satisfactory . Lee et al. [3 6] gave a local-search b ased algorithm that gives a (1 / 4 − ε ) -appro ximation to maximize F over th e polytope ind uced by a 4 If f is not assumed to be n on-neg ati ve , ev en t he unc onstraine d problem i s ina pproximabl e since decidi ng whether th e opti mum v alue is positi ve or zero requires an expone ntial number of queries. 5 A fun ction is modular if f ( A ) + f ( B ) = f ( A ∪ B ) + f ( A ∩ B ) for all A, B ⊆ N . If f i s modular the n f ( A ) = w 0 + P i ∈ A w i for some weight functi on w : N → R . 6 W e say that a polyt ope P is solvab le if one can do efﬁci ent linear optimizat ion ove r P . 2 ﬁxed numbe r of knap sack con straints. V o ndr ´ ak [5 0] obtained a 0 . 309 -ap proxim ation for max imizing F over a sing le matroid polytop e, and this ratio has been recently improved to 0 . 3 25 [4 3]. Howe ver, n o approx imation algorithm was known to maximize F over a general solvable polytope P . In terms o f rounding a fractio nal so lution x , a na tural s trategy to p reserve th e v alue of F ( x ) in expectation is to indepen dently rou nd each coordinate i up to 1 with probab ility x i and do wn to 0 otherwise. Howe ver , this roundin g strategy does n ot typically preserve the con straints imposed b y I . V arious dep endent rou nding schemes have been propo sed. It was shown in [6] that ”pipage roundin g” can be used to round solutions in the matroid polytope with- out losing in terms of the objective function F ( x ) ([15] achiev es the same via ”swap-ro undin g”). I n [ 33, 36, 4, 34], random ized rounding coupled with alteration was used for knapsack constraints. More recently , [15] showed concen- tration properties f or rounding in a sing le matroid p olytope when f is monoton e, and [52] sho wed con centration f or indepen dent rounding ev en when f is no n-mon otone. These led to a few additiona l results. D espite this p rogress, the “integrality gap ” of max { F ( x ) : x ∈ P } h as be en so far unkn own e ven when f is monoto ne and P the in tersection of two matroid polytope s. (W e rem ark that for intersections of matroids, combin atorial algorithms are known to yield good appro ximations [36, 37].) However , e ven for modular functions—i.e., classical linea r optimization—co mbining constraints such as matr oids and k napsack con straints has been dif ﬁcult, and no general result was known that ma tched the best boun ds o ne can get for them separately . In summ ary , previous results via the multiline ar relaxation were known only f or rath er restricted cases, both in terms of app roxim ately max imizing the multilinear extension, and in terms of ef fectiv ely rou nding fractional solutions. W e next describe the contributions of this p aper in this context. Our contribution at a high level: I n this paper we develop a g eneral framework for solvin g submodular maximization problem s of the for m ma x { f ( S ) : S ∈ I } , where f : 2 N → R + is su bmodu lar and I ⊂ 2 X is a downward-closed family of sets. Our framew ork consists of the follo wing componen ts. • Optimizing th e mu ltilinear relaxation: W e give the ﬁrst constant factor ap prox imation, with an additional n eg- ligible additiv e err or, fo r the p roblem max { F ( x ) : x ∈ P } where F is the multilinear extension o f any no n- negativ e sub modu lar fun ction, and P is any do wn-mono tone 7 solvable polytope. • Depend ent randomized r oundin g: W e propose a gener al ( depend ent) rand omized ro unding framew ork for mod- ular and sub modula r functions u nder in depend ence constrain ts via wh at we call co ntention r esolution s chemes ( C R schemes ). Rounding an approx imate maximizer of the relaxation max { F ( x ) : x ∈ P } via a C R scheme that is tailored to the gi ven constrain ts, leads to a solution with provable approximatio n guarantee. A key advan- tage of C R schemes is the ability to e asily combine C R schemes designed for d ifferent con straints into a C R scheme for the intersection of these constraints. • Contention r esolution schemes: W e present C R schemes fo r a variety o f packing constraints, includ ing k napsack constraints, matroid constraints, sparse pack ing systems, column-restricted packing constraints, and constraints imposed by an un splittable ﬂow problem in p aths and trees. Our C R scheme for the matro id polytope, which is p rovably optimal 8 , is obtained by explo iting a tight connectio n between C R schemes and the correlation gap [2] of the associated weighted r ank f unction s. Previously , in the con text of matr oids, an optimal C R scheme was only known fo r the uniform matroid of rank 1 [22, 23]. The abov e in gredients ca n be p ut together to o btain a relaxation an d r oundin g f ramework leadin g to a variety o f new results that we d iscuss in more detail in Section 2. W e summarize some of our results in T able 1 . 1.1 Maximizing the multilinear extension over a general polytope W e now give a more detailed description of o ur techn ical results and the gen eral framework. First, we gi ve a con- stant factor approxima tion fo r the problem max { F ( x ) : x ∈ P } , where F is the mu ltilinear e xtension of a non- monoto ne sub modu lar function f and P is a down-mo notone so lvable poly tope; the m onoton e case admits a (1 − 1 /e ) - approx imation [ 49] as we mentioned already . Th e condition of down-monoton icity of the polytop e is necessary for 7 A polyt ope P ⊆ [0 , 1] N is do wn-monotone if for all x , y ∈ [0 , 1] N , y ≤ x and x ∈ P implies y ∈ P . 8 W e w ould like to h ighlight that conte ntion resolutions schemes are ju st one way of rounding a fractiona l solution. Hence, the use of an optimal content ion resolutio n scheme does not imply that no better approximatio n factor can be obtained by a dif ferent procedure. 3 Constraint type Linear maximization Monotone submod. max . Non-ne gativ e submod. max. O (1) knapsacks [1 − ε [8, 26] ] [1 − 1 /e − ε [34] ] 0 . 325 [0 . 25 [36] ] k matroids & ℓ = O (1) knapsacks 0 . 6 k [Ω( 1 ( k + ℓ ) [28 , 29] ] 0 . 38 k [Ω( 1 ( k + ℓ ) [28 , 29] ] 0 . 19 k [Ω( 1 k + ℓ ) [28, 29] ] k - matchoid & ℓ -sparse PIP Ω( 1 k + ℓ ) Ω( 1 k + ℓ ) Ω( 1 k + ℓ ) Unsplittable ﬂow in paths an d trees [Ω(1) [14] ] Ω(1) Ω(1) T able 1: Appro ximation ratios for dif ferent types of constraints and objecti v e fun ctions. Results in square brac kets were previously kno wn. The ratios in the last column for non-mo notone fu nctions are based on a 0 . 325 approximation for maximizng the multilinear relaxation described in the confere nce version o f this p aper [16]; there is a 1 e − ε ≈ 0 . 367 -approximation from subseque nt work of [24] which results in improv ed bounds. the non-mono tone case; it follows from [50, 51] that no constant factor approximation is possible for the matroid base polytop e wh ich is not down-monoto ne. The main algorithmic te chnique for non-mon otone fun ctions has b een local search. Fractio nal local sear ch with additional ideas has been the tool to solve the continuous problem in special cases of polytop es [3 6, 50, 43]. Previous fractional local searc h methods ([36] and [50]) improved a current solution x by considerin g moves alon g a small number of co ordin ates of x . The analysis too k advantage o f th e combinatorial stru cture of the underlying con straint (knapsack s or matroids) which was sufﬁ ciently simple tha t s waps along a few coo rdinates suf ﬁced. Ho w do we obtain an algorithm that works for any polytope P ? A new insight: Our key hig h-level idea is simp le yet in sightful. Any point x ∈ P can b e written as a co n vex combinatio n of the vertices of P . W e view th e prob lem of max { F ( x ) : x ∈ P } as o ptimizing a submo dular function over the ground set co nsisting of the (exponen tially many) vertice s o f P (du plicated m any times in the limit). From this viewpoint we ob tain a ne w fractional local search procedure: g iv en a current point x , a local sw ap corresp onds to removing a v ertex in the con vex combination of x and adding a new v ertex o f P (with appr opriate scalar multipliers) . T o im plement th is ef ﬁciently we can use linear optimization over P . (W e remark that the con tinuous greedy algorithm for the mono tone case [ 49] can also be interpreted with this insight.) Our algorithms are deriv ed using the above h igh-level idea. W e note that when specialized to the matr oid p olytope or kna psack polytope which have combinato rial structure, our algorith ms become simpler and in fact resem ble previ- ous alg orithms. Ou r alg orithms an d pro ofs of app roxima tion guarantees are in fact simpler th an the pre viously given proof s for p articular polytope s [36, 50, 43]. W e pr esent two algorithms f ollowing this idea. T he ﬁr st a lgorithm is close in spir it to the local-search a lgorithm of Lee et al. for knapsack co nstraints [36] and g iv es a 0 . 25 - appro ximation. This algo rithm, despite ha ving a worse approx imation guar antee then the seco nd one we present, allo ws u s to f urther e xplain an d f ormalize the ab ove high - lev el idea in a clean w ay . The second algorithm uses s ome ideas of [50] for th e case of a matroid po lytope and gi ves a 0 . 309 - appro ximation with respect to the best inte ger solution in P . W e would like to mention that sub sequently to the co nferenc e version of this paper, Feld man et al. [2 4] presented an improved algorithm to max imize th e multilinear extension , lea ding to an ( e − 1 − ǫ ) ≈ 0 . 3 6 7 -app roxima tion with respect to the best in teger solution. Their algorithm is an adaptation of the continuous greed y algorithm [49]. T he confere nce version of this p aper [16] contained a thir d and mu ch more in volved algorithm (generalizin g the simulated annealing approach of [43]) that gives a 0 . 32 5 -app roxima tion, again with respect to the b est integer solution in P . For conciseness, and in view of the recent results in [ 24], we do not includ e this third algorithm in this pap er, and concentr ate on the ﬁrst two algorithms mentioned in the preceding pa ragraph , which allow us to d emonstrate the main new algorithmic insights. W e summ arize our results in the follo wing theorem. Theorem 1. 1. Let f be a nonnegative submodular function and P ⊆ R n be a solvable down-mon otone po lytope satisfying tha t there is a λ ∈ Ω( 1 p oly( n ) ) such tha t fo r each coo r dinate i ∈ [ n ] , λ · e i ∈ P . then ther e is a (0 . 2 5 − o (1)) - appr oximation algorithm for the pr ob lem max { F ( x ) : x ∈ P } wher e F is the multilinear extension of f . Ther e is also an alg orithm for th is pr oblem which returns a solu tion y ∈ P of va lue F ( y ) ≥ 0 . 309 · max { F ( x ) : x ∈ P ∩ { 0 , 1 } N } . W e r emark that a known limit on the approximability of max { F ( x ) : x ∈ P } is an information theor etic hardness of 0 . 478 - appro ximation in the value oracle m odel, ev en in the special case of a matroid polytope [43]. 4 1.2 Contention r esolution schemes W e sho w that a certain natural class of round ing schemes that we call contention r esolutio n sc hemes ( C R schemes ) provides a useful and gen eral framework for rounding fractiona l solutions under submo dular objectiv e functions. For a grou nd set N , let P I be a co n vex relaxation of the constrain ts imp osed by I ⊆ 2 N , an d let x ∈ P I . From the deﬁnition of F , a natura l strategy to ro und a point x is to independ ently ro und the coo rdinates; howe ver , this is unlikely to preserve th e constraints im posed b y I . Let R ( x ) ⊆ N be a random set obtain ed by includ ing eac h element i ∈ N indepen dently with prob ability x i . The set R ( x ) is n ot n ecessarily feasible. W e would like to remove (rando mly) some elements fr om R ( x ) , so that we obtain a feasible set I ⊆ R ( x ) . The property we would like to achieve is th at every element i appears in I with probab ility at least cx i for som e parameter c > 0 . W e call such a schem e “ c -balanced co ntention resolution ” for P I . W e stress th at a c - balanced C R scheme needs to work for all x ∈ P I . Howe ver , often, stron ger sche mes—i.e. with larger values for c —can be ob tained if th ey on ly need to work for all points in a scaled-down version bP I = { b · x | x ∈ P I } of P I , where b ∈ [0 , 1] . Suc h schemes, which we call ( b, c ) -balanced schemes, will prove to be useful when combinin g C R schemes for dif ferent constraints as we will discuss in Section 1.3. Below is a for mal deﬁnition of C R schemes. Let supp ort ( x ) = { i ∈ N | x i > 0 } . Deﬁnition 1.2. Let b, c ∈ [0 , 1] . A ( b, c ) -bala nced C R scheme π for P I is a p r oce dur e that fo r every x ∈ bP I and A ⊆ N , returns a random s et π x ( A ) ⊆ A ∩ suppo rt ( x ) and satisﬁes the following pr o perties: (i) π x ( A ) ∈ I with pr o bability 1 ∀ A ⊆ N , x ∈ bP I , and (ii) for all i ∈ support ( x ) , Pr[ i ∈ π x ( R ( x )) | i ∈ R ( x )] ≥ c ∀ x ∈ bP I . The scheme is said to be monoto ne if Pr[ i ∈ π x ( A 1 )] ≥ Pr[ i ∈ π x ( A 2 )] whenever i ∈ A 1 ⊆ A 2 . A (1 , c ) -b alanced C R scheme is also called a c -balanced C R scheme . The scheme is deterministic if π is a deterministic a lgorithm (hence π x ( A ) is a single set instead o f a distribution). It is o blivious if π is deterministic and π x ( A ) = π y ( A ) for all x , y an d A , tha t is, the ou tput is indepen dent of x a nd only depe nds on A . The scheme is efﬁciently implementable if π is a po lynomial- time a lgorithm that given x , A outpu ts π x ( A ) . W e emphasize that a C R scheme is deﬁn ed with re spect to a spe ciﬁc po lyhedra l relaxation P I of I . Note that o n the left-hand side o f condition (ii) fo r a C R sch eme, the probability is with r espect to two ra ndom sources: ﬁrst the set R ( x ) is a random set, and seco nd, the procedure π x is typically randomized. W e note th at a ( b, c ) -b alanced C R scheme π can easily be transforme d into a bc -balan ced C R scheme; details are gi ven in Section 4. The theorem below h ighlights the utility of of C R scheme s; when roun ding via m onoto ne co ntention resolution schemes, one can claim an expec tation bound for submod ular functions. A similar theorem was shown in [4] for monoto ne functions. W e state and prove ours in a form s uitable for our context. Theorem 1.3 . Let b, c ∈ [0 , 1] , a nd let f : 2 N → R + be a non-n e gative submodular functio n with multilinear r elaxation F , an d x ∈ b · P I , wher e P I is a co n vex r elaxation for I ⊆ 2 N . Fu rthermor e, let π b e a mo noton e ( b, c ) -ba lanced C R sc heme for P I , and let I = π x ( R ( x )) . If f is monotone then E [ f ( I )] ≥ c F ( x ) . Furthermore , t her e is a function η f : 2 N → 2 N that depend s on f and can be evaluated in linear time, such tha t even for f non- monoto ne E [ f ( η f ( I ))] ≥ c F ( x ) . As we will see in Section 4, the functio n η f can be chosen to always return a subset of its argument. W e therefo re call it a pruning operation . W e observe that several previous rounding pro cedure s for packing (and also co vering) problems rely on the well- known technique o f alteration of a set o btained v ia indep enden t rounding and are examples of C R schemes ( see [45, 5, 9, 14, 4]). However , these schemes are typically oblivious in th at they do not depend on x itself (other than in picking the random set R ), and the alteratio n is determin istic. Our d eﬁnition is inspire d b y the “fair co ntention resolution schem e” in [22, 23] which c onsidered the special case of contention for a single item. The de penden ce on x as well as ran domizatio n is necessary (even in this case) if we want to ob tain an optimal scheme. One key question 5 to consider is whether some g iv en d own-monoton e polytope P I admits a “go od” ( b, c ) -balanced CR scheme, which correspo nds t o having v alues of b and c that are as close to 1 as possible. One n atural w ay to ap ply a ( b , c ) -balan ced CR scheme to a point ˆ x ∈ P I that approximately max imizes F is as follows. In a ﬁrst step we scale down ˆ x to obtain x = b · ˆ x . By non-negativity and concavity of F along non -negative directions one obtains F ( x ) ≥ b · F ( ˆ x ) . Applying a ( b, c ) -balanced CR scheme π to x leads to a s et I = π ( R ( x )) which, according to Th eorem 1.3 , sati sﬁes f ( I ) ≥ cF ( x ) ≥ cbF ( ˆ x ) . This also highlights the motiv ation why we w ant to hav e b and c as close to 1 as possible. As we will show , many natural constrain t systems admit good ( b, c ) -b alanced CR sch emes, inclu ding matroid constraints, knapsack constraints, and a variety of packing integer pr ogram s. In particular, to deal with the rather general class of matroid constrain ts, we exploit a clo se conn ection between the existence of CR schemes and a recently introdu ced concept, called corr elation gap [53]. Contention resolution via correlation gap and an optima l scheme for matroids: Until recently ther e was no contention r esolution s cheme for the matroid po lytope; an optimal ( b, 1 − e − b b ) -balanced scheme was pre viously known for the very special ca se of the unifo rm matro id o f rank o ne [22, 2 3]. W e note that the recent work of Cha wla et a l. [11, 1 2] imp licitly co ntains a ( b, 1 − b ) -balance d determ inistic scheme fo r m atroids; the ir mo tiv ation for co nsidering this notion was mechanism design. In this paper we de velop a n optim al scheme for an arbitrary matroid 9 . Theorem 1.4. There is an optimal ( b, 1 − e − b b ) -balanc ed contention res olution scheme for any matr oid polytop e. Mo r e- over the scheme is monotone and e fﬁciently implementable. The main idea in pr oving the preceding theorem is consider a random ized C R scheme and view it abstractly as a conv ex combination of d eterministic C R schem es. Th is allows , via LP duality , to sho w that the b est contention resolution schem e fo r a constraint system is related to the notion of correlation gap for weighted rank functions o f the underly ing constraint. W e reiterate that th e scheme depend s o n the frac tional solution x that we wish to roun d; the alter ation of the rando m set R ( x ) is itself a randomized pr ocedur e that is tailored to x , and is found by so lving a linear progr am. W e are inspired to make the general co nnection to correlation gap due to the recent work of Y an [53]; he applied a similar idea in the co ntext of greedy posted- price ordering schemes fo r Bayesian mechanism design, improving the bounds of [11, 12]. 1.3 A framework f or r o unding via contention resolution scheme s W e now describe our fram ew ork for the problem max S ∈I f ( S ) . The fr amew ork a ssumes the following: (i) there is a polyno mial-time value oracle f or f , (ii) ther e is a solvable d own-monoton e poly tope P I that con tains the set { 1 S | S ∈ I } , and (iii) there is a m onoto ne c -b alanced contention resolution scheme π for P I . T hen we have th e following simple algorithm: 1. Using an appro ximation algorithm, obtain in p olynom ial tim e a point x ∗ ∈ P I such that F ( x ∗ ) ≥ α · max { F ( x ) | x ∈ P I ∩ { 0 , 1 } N } ≥ α · max S ∈I f ( S ) . 2. Round the point x ∗ using the CR scheme π to ob tain I = π x ∗ ( R ( x ∗ )) , and return its prune d versio n η f ( I ) . Theorem 1.5. The preceding framework gives a r andomized ( α c ) -appr oximation algorithm for max S ∈I f ( S ) , when- ever f is non-negative submodular , α is the appr oximation ratio for max { F ( x ) | x ∈ P I ∩ { 0 , 1 } N } and P I admits a monoton e c - balan ced C R sc heme. If f is monoton e th en the pruning step is not n eeded. If f is modular then the r atio is c and the C R s cheme is not even constrained to be mo notone. Pr oo f. W e have F ( x ∗ ) ≥ α O P T with O P T = max S ∈I f ( S ) . Theo rem 1. 3 shows that E [ f ( η f ( I ))] ≥ cF ( x ∗ ) , hence E [ f ( η f ( I ))] ≥ αc O P T . If f is monoton e, the p runin g step is not required by Theorem 1.3. For modular f , F ( x ) is a lin ear fun ction, and h ence α = 1 can b e obtain ed by linear programmin g. Mor eover , if F ( x ) is a linear function, then b y linearity o f expec tation, E [ f ( I )] ≥ cF ( x ∗ ) w ithout any mon otonicity assump tion on the scheme. 9 W e also describe the ( b, 1 − b ) scheme i n Section 4.4 for completeness. This scheme is simpler and computational ly adva ntageo us when compared to the optimal scheme. 6 For non-mon otone submodu lar functions, Theorem 1.1 gives α = 0 . 309 ; the cu rrently best known approx imation is ( 1 e − ε ) ≃ 0 . 367 due to [24]. For monoton e su bmodu lar functions an op timal bound of α = 1 − 1 e is giv en in [49]. Combining schemes for different constra ints: W e are par ticularly interested in the case when I = ∩ h i =1 I i is the intersection of several different independ ence systems on N ; each system corr esponds to a different set of constraints that we would like to impose. Assuming that we can apply the above framew ork to each I i separately , we c an obtain an algorithm for I as follows. Lemma 1.6. Let I = ∩ h i =1 I i and P I = ∩ i P I i . Suppose eac h P I i has a mono tone ( b, c i ) -balanc ed CR scheme. Then P I has a monoton e ( b, Q i c i ) -balanc ed CR scheme. In the special case that eac h element of N participates in at most k constraints an d c i = c for all i then P I has a monoton e ( b, c k ) -balanc ed C R sc heme. Mor eover , if the scheme fo r each P I i is implementable in polynomial time then the c ombined scheme for P I can be implemented in polynomial time. Therefo re, we can p roceed as follows. Le t P I i be a polytope that is the relaxation for I i . In othe r w ords { 1 S : S ∈ I i } is contained in P I i . Let P I = ∩ i P I i . It follows that { 1 S : S ∈ I } is contain ed in P I and also that th ere is a polyn omial-time separatio n o racle for P I if the re is one for each P I i . Now sup pose there is a monoto ne ( b, c i ) - balanced con tention resolu tion scheme for P I i for som e co mmon cho ice of b . I t fo llows from Lemma 1.6 that P I has a mono tone ( b, Q i c i ) -balanced contention resolution sch eme, which can b e transform ed into a ( b Q i c i ) -balanced scheme fo r P I . W e can then ap ply Th eorem 1.5 to obtain a randomized ( αb Q i c i ) -appro ximation for max S ∈I f ( S ) where α depends on whether f is modular, mon otone submodular or non-mono tone submo dular . In this pa per we f ocus on the fram ew ork with a small list of h igh-level applications. W e have not attempted to optimize for the b est possible app roxim ation fo r special cases. W e add tw o remark s that are usefu l in augmentin g the framework. Remark 1 .7. Whenever the r oundin g step of o ur framework is performed by a C R scheme that wa s o btained fr om a ( b, c ) -ba lanced CR scheme—in particula r in the co ntext mentioned above when co mbining C R schemes for differ ent constraints—we can often str en gthen the pr ocedure a s fo llows. In stead o f ap pr o ximately solving max x ∈ P I F ( x ) , we c an appr oxima tely solve max y ∈ bP I F ( y ) to obtain y ∗ , and then dir ectly a pply the ( b, c ) -balan ced scheme to y ∗ , without transforming it ﬁrst to a bc -b alanced scheme. This may be a dvantageous if the pr o blem max y ∈ bP I F ( y ) admits a direct appr o ximation better than one ob tained by scaling fr om max y ∈ P I F ( y ) . A usefu l fact her e is that th e continuo us greedy algo rithm fo r m onoton e submod ular fun ctions [5 0, 7] ﬁnds fo r every b ∈ [0 , 1 ] a point y ∗ ∈ bP I such that F ( y ∗ ) ≥ (1 − e − b ) max x ∈ P I F ( x ) . This is indeed a str onger guarantee tha n the one ob tained by ﬁrst applying the contin uous gr eed y to P I to obtain x ∗ , and then used the sca led-down version b x ∗ , which leads to a guarantee of only F ( b x ∗ ) ≥ bF ( x ∗ ) ≥ b (1 − e − 1 ) max x ∈ P I F ( x ) . Remark 1.8. A non-n e gative submodu lar set function f is also sub additive, that is, f ( A ) + f ( B ) ≥ f ( A ∪ B ) . In some setting s when c onsidering the p r o blem ma x S ∈I f ( S ) , it ma y b e advan tageous to partition the give n gr ound set N into N 1 , . . . , N h , separately solve the pr oblem on each N i , and then r etu rn the best of these solutions. This loses a factor of h in the a ppr oximation but one ma y be a ble to o btain a g ood C R scheme for ea ch N i separately while it may not be straightforwar d to o btain one for the entir e set N . An application of the technique m entioned in Remark 1.8 can be found in Section 4.8, wher e we use it in the context of column-restricted packing constraints. Organization: The rest of th e paper is divided into three parts. Some illustrative applications of our framework are discussed in Section 2. Con stant factor appro ximation algorithms for maxim izing F over a solvable p olytope are described in Sectio n 3. Section 4 d iscusses the construction o f C R schemes. This include a discussion of th e connectio n between co ntention resolution scheme s and correlation g ap and its u se in deriving optimal schemes for matroids. Furthermore, in the same section, we present C R schemes for knapsack con straints, sparse packing systems, and UFP in paths and trees. 7 2 A ppl ications In this section we brieﬂy outline some concrete results that can be obtained via our fra mew ork. The meta-problem we are interested in so lving is max S ∈I f ( S ) where I is a d ownward-closed family over the given ground set N a nd f is a non-negative sub modu lar set f unction over N . Many interesting problems can be cast as special cases depending o n the choice o f N , I and f . In order to app ly t he framew ork an d obtain a polynom ial-time appro ximation alg orithm, we need a solv able relaxation P I and a correspondin g ( b, c ) -bala nced C R sch eme. Note that t he framework is essentially indifferent to f as long as we ha ve a p olynom ial-time value oracle for it. W e therefore focus on som e broad classes of constraints an d correspon ding natural polyhed ral relaxations, and discuss C R schemes that c an be o btained for them . These schemes are formally described in Section 4. Matr oid s and m atchoids: L et M = ( N , I ) be a m atroid co nstraint on N . A natural candidate f or P I is the in tegral matroid po lytope { x ∈ [0 , 1] n | x ( S ) ≤ r ( S ) , S ⊆ N } wher e r : 2 N → Z + is the rank functio n of M . W e develop an optim al (1 − 1 /e ) -balan ced C R scheme for the matr oid polytop e. Mor e generally , for any b ∈ (0 , 1] we design a ( b, 1 − e − b b ) -balanced C R schem e, which lend s itself well to combinatio ns with other con straints. The C R schem e f or the matroid polytope extend s via Lem ma 1.6 to the case when I is induce d b y the intersection o f k matroid con straints o n N . A more general result is obtain ed by conside ring k -unif orm matchoids, a commo n generalizatio n o f k -set pack ing and intersectio n of k matr oids [38], d eﬁned as follows. Let G = ( V , N ) be a k - unifor m hypergrap h; we associate the edges o f th e hyp ergraph with our g round set N . For each v ∈ V , there is a matroid M v = ( N v , I v ) over N v , set of hyp eredges in N that co ntain v . This induces an independence family I on N where I = { S ⊆ N | S ∩ N v ∈ I v , v ∈ V } . k - unifor m match oids generalize the intersection o f k matro ids in that they allow many m atroids in the intersection as long as a given element of the grou nd set par ticipates in at most k of them. A natural solv able r elaxation for I is the in tersection o f the matroid poly topes at each v . V ia the C R schem e f or the single m atroid and Lemma 1 .6 we o btain a ( b , ( 1 − e − b b ) k ) -balanced C R scheme for any b ∈ (0 , 1] for k -unifo rm matchoids. The choice of b = 2 k +1 giv es a 2 e ( k +1) -balanced C R sch eme for every k -unifo rm ma tchoid. Knapsack / linear packing constraints: Let N = { 1 , 2 , . . . , n } . Gi ven a non- negativ e m × n matrix A and n on- negativ e vector b , let I = { S | A 1 S ≤ b } where 1 S is th e in dicator vector o f set S ⊆ N . It is easy to see that I is an ind epend ence family . A natural LP relax ation fo r th e p roblem is P I = { x | A x ≤ b , x ∈ [0 , 1] n } . The width of th e system of inequalities is deﬁned as W = ⌊ min i,j b i / A i,j ⌋ . Some special cases of interest are (i) A is a { 0 , 1 } - matrix, ( ii) A is colum n-restricted, that is, all non-zero entr ies in each column are the same and ( iii) A is k -colu mn sparse, that is at mo st k non-zero entries in each co lumn. Several combin atorial prob lems can be captur ed by these, s uch as m atchings and ind epende nt sets in graphs and hypergrap hs, knapsack an d its variants, and maximum throug hput rou ting pro blems. However , the maximum in depend ent set problem in graphs, which is a spec ial case as mentioned , d oes n ot allo w a n 1 − ε -appro ximation for any ﬁxed ε > 0 , unless P = NP [3 0]. Therefor e attention has focused on restricting A in various ways and obtaining upper bounds on the integrality g ap of the relaxation P I when the objective fun ction is linear . Sev eral of these results are based on rando mized rounding of a fractional solution and one can interpre t the rounding alg orithms as C R schemes. W e con sider a few such re sults below . • For a co nstant numb er of knap sack co nstraints ( m = O (1) ), by g uessing an d enumer ation tricks, one can “effecti vely” get a (1 − ε, 1 − ε ) -balan ced C R scheme for any ﬁxed ε > 0 . • When A is k -co lumn sp arse, there is a ( b, 1 − 2 k b ) -b alanced C R schem e. If A has in addition wid th W ≥ 2 , there is a ( b, 1 − k (2 eb ) W − 1 ) C R schem e for any b ∈ (0 , 1) . These results follow fro m [4]. • When A is a { 0 , 1 } -matrix ind uced b y th e p roblem of ro uting u nit-deman d paths in a capacitated path o r tre e, there is a ( b, 1 − O ( b )) C R scheme implicit in [5, 9, 14]. This can be e xtended to the unsplittable ﬂo w problem (UFP) in capacitated paths and trees via group ing and scalin g techniques [31, 14, 13]. Section 4 has formal details of the claimed C R schemes. Th ere are other rounding schemes in the literature f or packing pr oblems, typically developed for linear fun ctions, that can be reinterpreted as C R schemes. Our f ramework can then b e used t o ob tain algo rithms for no n-negative submo dular set function s. See [10] for a re cent and illu minating example. Ap proximation algorit hms. The C R schemes mentioned ab ove whe n instantiated with suitable par ameters and plugged into ou r gener al framework yield several ne w random ized p olyno mial-time appro ximation algorithms fo r 8 problem s of the form max S ∈I f ( S ) , wh ere f is non -negative subm odular . W e r emark that these results ar e for some- what abstract p roblem s and one can obtain more co ncrete results by specia lizing them an d improving the constants. W e ha ve not attempted to do so in this paper . • If I is the intersection of a ﬁxed num ber of knapsack constraints, we achiev e a 0 . 309 - appro ximation, im proving the (0 . 2 − ε ) -app roximatio n from [36] and a rec ent (0 . 25 − ε ) -approx imation [34]. This is obtained via the (1 − ε, 1 − ε ) -balance d C R scheme f or a ﬁxed number of knapsack constraints. • If I is the intersection of a k -unif orm m atchoid and ℓ knap sack c onstraints with ℓ a ﬁxed constant, we ob tain an Ω( 1 k ) -appro ximation (constant in depend ent of ℓ ) , whic h imp roves the bou nd of Ω( 1 k + ℓ ) from [ 28]. W e remar k that th is is a ne w r esult even for linear o bjective functions. W e o btain this by ch oosing b = Ω(1 /k ) and using the ( b, ( 1 − e − b b ) k ) -balanced C R scheme for k - unifor m m atchoids and the (1 − ε, 1 − ε ) -balanced C R scheme for a ﬁxed number of knapsack constraints (this requires a separate preprocessing step ). • If I is the in tersection of a k -unif orm matcho id and an ℓ -sparse knap sack constraint s ystem of width W , we give an Ω( 1 k + ℓ 1 /W ) -appro ximation, imp roving the Ω( 1 kℓ ) a pprox imation f rom [2 8]. This follows by combining th e C R schem es for k -unif orm match oid an d ℓ -co lumn s parse packing constrain ts with a choice of b = Ω( 1 k + ℓ 1 /W ) . • W e ob tain a constant factor app roxima tion for maxim izing a non-negative subm odular functio n of r outed re- quests in a capacitated path or tree. Pre viously a n O (1) a pprox imation was known for linear functions [ 5, 9, 14, 13]. 3 Solving the mul tilinear r elaxation f or non-negative submodular functions In this section, we a ddress the question of solvin g the problem max { F ( x ) : x ∈ P } wher e F is the m ultilinear extension of a sub modular function. As we alre ady mention ed, due to [4 9, 7], ther e is a (1 − 1 / e ) -appr oximation for the problem max { F ( x ) : x ∈ P } whenever F is the multilinear exten sion o f a mo noton e sub modular fu nction and P is any solvable po lytope. Here, we co nsider the maximization of a possibly non- monoto ne su bmodu lar function over a down-mo notone solvable polytope. W e assum e in the follo wing that P ⊆ [0 , 1 ] N is a down-monotone solvable polytop e and F : [0 , 1] N → R + is the multilinea r extension of a submodu lar fu nction. W e present two algorithm s for this problem . As we no ted in the introdu ction, there is no constant-factor a pprox imation for maxim izing non- monoto ne submodular functio ns ov er general—i.e. , not n ecessarily down-mon otone—so lvable polytopes [50]. The approx imation th at can b e achieved f or matroid base po lytopes is prop ortiona l to 1 − 1 /ν whe re ν is the fra ctional packing number of bases (see [5 0]), and in fact th is trade-off generalizes to ar bitrary solv able po lytopes; we d iscuss this in Appendix A. 3.1 Continuous local-search Here we p resent our ﬁrst algo rithm for th e problem ma x { F ( x ) : x ∈ P } . W e rem ark that in the special case of multiple knapsack constraints, this algorithm is equiv alent to th e algorithm of [36]. First we consider a natural local-search algorithm that tries to ﬁnd a local optimum for F in th e polytope P . For a continuo us function g d eﬁned over a conve x set C ⊆ R n , a point x ∈ C is a local optimum (in p articular, a m aximum ), if g ( x ) ≥ g ( x ′ ) for all x ′ ∈ C in a n eighbor hood of x . If g is d ifferentiable ov er C , a ﬁrst-order ne cessary cond ition for x to be a loc al max imum is that ( y − x ) · ∇ g ( x ) ≤ 0 for all y ∈ C . If g is in ad dition a concave functio n th en this is in fact sufﬁcient for x to be a g lobal max imum. Howe ver , in general th e ﬁrst-o rder n ecessary con dition is not sufﬁcient to gu arantee e ven a local optim um. Although suf ﬁcient conditions based on second-order partial d eriv ati ves exist, it is non- trivial to ﬁnd a lo cal o ptimum or to certify that a gi ven point x is a local o ptimum. Our alg orithms and analysis rely on ly on ﬁnd ing a p oint which satisﬁes (ap proxim ately) the ﬁrst-order necessary condition. Hence, this po int is n ot n ecessarily a local optimu m in the classical sense. Nev ertheless, for n otational co n venience we refer to any su ch p oint as a local o ptimum (som etimes such a point is ref erred to as a con strained critical po int). A simple high-level p rocedu re to ﬁnd such a local optimum for F ( x ) in P —which does no t co nsider implemen tability—is the following. W e will sub sequently discuss how to ob tain an efﬁcient version of th is hig h-level app roach that re turns an approx imate local optimum. 9 Algorithm 3.1. Con tinuou s local sear ch: Initialize x := 0 . As long as ther e is y ∈ P such that ( y − x ) · ∇ F ( x ) > 0 , move x con tinuously in the dir e ction y − x . If there is n o such y ∈ P , return x . This alg orithm is similar to gradient descen t (o r rather ascent), and without considering pr ecision and conver gence issues, it w ould be eq uiv alent to it. The im portanc e of the particular for mulation that we stated here will becom e mor e clear when we discretize the algorith m, in order to argue that it terminates in polynomia l time and ach iev es a solution with suitable properties. The objectiv e function F is no t conca ve; ho we ver , subm odularity implies that along any non-negativ e direction F is co ncave (see [49, 7]). Th is leads to the following basic lemma and its co rollary abo ut lo cal o ptima that we rely on in the analysis of our algorithms. In the following x ∨ y denotes the vector obtained b y taking the coordinate -wise maximum of the vectors x and y ; and x ∧ y de notes the vector obtained by taking the coor dinate-wise minimum. Lemma 3.2. F or any two points x , y ∈ [0 , 1 ] N and the multilinear e xtension F : [0 , 1] N → R of a submodular function, ( y − x ) · ∇ F ( x ) ≥ F ( x ∨ y ) + F ( x ∧ y ) − 2 F ( x ) . Pr oo f. By subm odularity , F is con cave alo ng any line with a nonnegativ e d irection vector, such a s ( x ∨ y ) − x ≥ 0 . Therefo re, F ( x ∨ y ) − F ( x ) ≤ (( x ∨ y ) − x ) · ∇ F ( x ) , and similarly F ( x ∧ y ) − F ( x ) ≤ (( x ∧ y ) − x ) · ∇ F ( x ) , because of the conca vity of F along direction ( x ∧ y ) − x ≤ 0 . Adding up these two inequ alities, we g et F ( x ∨ y ) + F ( x ∧ y ) − 2 F ( x ) ≤ (( x ∨ y ) + ( x ∧ y ) − 2 x ) · ∇ F ( x ) . It remains to observe that ( x ∨ y ) + ( x ∧ y ) = x + y , which proves the lemm a. Corollary 3 .3. If x is a local optimum in P , i.e. ( y − x ) · ∇ F ( x ) ≤ 0 for all y ∈ P , then 2 F ( x ) ≥ F ( x ∨ y ) + F ( x ∧ y ) for any y ∈ P . 3.2 Discre tized local search What follows is a discretization of Algorith m 3 .1, w hich is the one we actu ally use in our framework. Let M = max { f ( i ) , f ( N − i ) : i ∈ N } . Notice that M is an upper bou nd on the maximum absolute m arginal value o f any element, i.e., M ≥ max S,i | f S ( i ) | = max { f ( i ) − f ( ∅ ) , f ( N − i ) − f ( N ) : i ∈ N } . By subadditivity , we have | f ( S ) | ≤ M n fo r all S . It can b e also veriﬁed easily that | ∂ F ∂ x i | ≤ M an d | ∂ 2 F ∂ x i ∂ x j | ≤ 2 M for all i, j (see [50]). W e pick a p arameter q = n a for some sufﬁciently large constant a > 3 and maintain a conve x co mbinatio n x = 1 q P q i =1 v i , wh ere v i are ce rtain po ints in P ( without lo ss o f g enerality vertices, with p ossible r epetition). Each discrete step co rrespon ds to replacing a vector in the c onv ex com bination by ano ther . Instead of the gr adient ∇ F ( x ) , we use an estimate o f its co ordinate s ∂ F ∂ x i by random sam pling. W e use the following lemm a to contro l the errors in our estimates. Lemma 3.4. Let ˜ F ( x ) = 1 H P H h =1 f ( R h ) where R h is a rand om set s ampled independen tly with pr o babilities x i . Let H = n 2 a +1 , δ = M /n a − 1 and M = max { f ( i ) , f ( N − i ) : i ∈ N } . Then the pr obability that | ˜ F ( x ) − F ( x ) | > δ is at most 2 e − n/ 8 . Pr oo f. Let us deﬁne X h = 1 2 M n ( f ( R h ) − F ( x )) , a ran dom variable b ound ed by 1 in ab solute value. By deﬁnition, E [ X h ] = 0 . By the Chernoff bound, Pr[ | P H h =1 X h | > t ] < 2 e − t 2 / 2 H (see Theor em A.1.16 in [3]). W e set H = n 2 a +1 and t = 1 2 n a +1 , and obtain Pr[ | ˜ F ( x ) − F ( x ) | > M /n a − 1 ] = Pr[ | P H h =1 X h | > 1 2 n a +1 ] < 2 e − n/ 8 . Giv en estimates of F ( x ) , we can also estimate ∂ F ∂ x i = F ( x ∨ e i ) − F (( x ∨ e i ) − e i ) = E [ f ( R + i ) − f ( R − i )] . The above imp lies the following bo und. Corollary 3 .5. Let δ = M /n a − 1 . If the total number o f e valuation s of F and ∂ F ∂ x i is bound ed by n b and each estimate is co mputed ind ependen tly using n 2 a +1 samples, th en with pr obab ility at lea st 1 − O ( n b e − n/ 8 ) all the estimates are within ± δ additive err or . 10 The algorithm works as follows. The input to the algo rithm is a submod ular fu nction f given by a v alue o racle, and a polytope P given by a separa tion oracle. Algorithm 3.6. F ractional loca l searc h. Let q = n a , δ = M / n a − 1 . Let x := 1 q P q i =1 v i , and initialize v i = 0 for all i . Use estimates ˜ ∇ F ( x ) of ∇ F ( x ) within ± δ in each co or dinate. As long as ther e is y ∈ P such that ( y − x ) · ˜ ∇ F ( x ) > 4 δ n (which can be fo und b y lin ear p r ogramming), we modify x := 1 q P q i =1 v i by replacing one of the vectors v i in the linear combina tion by y , so that we maximize F ( x ) . If ther e is no such y ∈ P , return x . Lemma 3.7. Algorithm 3.6 terminates in polyno mial time with high pr obab ility . Pr oo f. W e sho w that if all e stimates of ∇ F compute d d uring the algo rithm are within ± δ in each coord inate—which happen s with high probability—th en the algo rithm terminates in polyn omial time. This implies the lemma sinc e with high p robability , we have that a polynomia l number o f estimate s o f ∇ F are ind eed all within ± δ in each co ordinate. Hence, we assume in the following th at all estimates ˜ ∇ F of ∇ F ar e within ± δ . In eac h step, the algorithm continues on ly if it ﬁn ds y ∈ P such that ( y − x ) · ˜ ∇ F ( x ) ≥ 4 δ n . Since ˜ ∇ F approx imates ∇ F within ± δ in each coordina te, this m eans that ( y − x ) · ∇ F ( x ) ≥ 3 δ n . D enote by x ′ a random vector that is o btained by replacing a random v ector v i by y , in the linear combina tion x = 1 q P q i =1 v i . The expected effect of this ch ange is E [ F ( x ′ ) − F ( x )] = 1 q q X i =1  F  x + 1 q ( y − v i )  − F ( x )  = 1 q 2 q X i =1 ( y − v i ) · ∇ F ( ˜ x i ) where ˜ x i is some point on the lin e between x and x + 1 q ( y − v i ) , following from the mean-value th eorem. Since q = n a and the second partial deriv ati ves of F are bound ed by 2 M , we get by standard bounds that ||∇ F ( ˜ x i ) − ∇ F ( x ) || 1 ≤ n 2 q · 2 M = 2 M n a − 2 = 2 δ n . Using also the fact that y − v i ∈ [ − 1 , 1] n , E [ F ( x ′ ) − F ( x )] ≥ 1 q 2 q X i =1 (( y − v i ) · ∇ F ( x ) − 2 δ n ) = 1 q (( y − x ) · ∇ F ( x ) − 2 δ n ) ≥ 1 q · δ n using the fact that ( y − x ) · ∇ F ( x ) ≥ 3 δ n . Theref ore, if we exchange y f or the vertex v i that maximizes ou r gain, we gain at least F ( x ′ ) − F ( x ) ≥ 1 q δ n = M n 2 a − 2 . Also we have the trivial b ound max F ( x ) ≤ nM ; th erefore the numbe r of steps is boun ded by n 2 a − 1 . Lemma 3.8. If x is the output of Algorithm 3.6, then with high pr o bability 2 F ( x ) ≥ F ( x ∨ y ) + F ( x ∧ y ) − 5 δ n for every y ∈ P . Pr oo f. If the algorithm terminates, it means that for every y ∈ P , ( y − x ) · ˜ ∇ F ( x ) ≤ 4 δ n . Considering the accuracy of ou r estimate o f the gr adient ˜ ∇ F ( x ) (with hig h probab ility), this mean s that ( y − x ) · ∇ F ( x ) ≤ 5 δ n . By Lemma 3.2, we have ( y − x ) · ∇ F ( x ) ≥ F ( x ∨ y ) + F ( x ∧ y ) − 2 F ( x ) . This proves th e lemma. 3.3 Repeated local search : a 0.25-appr oximation Next, we show that ho w to design a 0 . 25 -ap prox imation to the multilinear optimization p roblem using two runs of the fractional local-search algorithm . T he following is ou r algorithm. Algorithm 3.9. Let x b e the ou tput of Algorithm 3.6 on the po lytope P . Deﬁ ne Q = { y ∈ P : y ≤ 1 − x } an d let z be the output of Algorithm 3.6 on the polytop e Q . Return the better of F ( x ) and F ( z ) . W e use the following pro perty of the multilinear extension of a submodula r function . L et u s replace each c oordin ate by a [0 , 1] interval and let us re present a certain value x i of the i ’th coordinate by a subset o f [0 , 1] of the correspond ing measure. 11 Deﬁnition 3.10. Let X ∈ L N , wher e L den otes the set of all measurable su bsets of [0 , 1] . W e say that X r epr e sents a vector x ∈ [0 , 1] N , if X i has measur e x i for each i ∈ N . From a ”discrete poin t of view”, we can imagine that each co ordina te is repla ced by some large number of elemen ts M and a value of x i is repr esented by any subset of size M x i . This can be carr ied out if all the vectors we work with are r ational. In the fo llowing, we consider fu nctions on subsets of th is n ew gro und set. W e sho w a natural property , namely that a function der iv ed f rom the multilinear extension of a submod ular function is again submodular . (An analogo us p roperty in the discrete case was proved in [4 0, 36].) Lemma 3.1 1. Let F : [0 , 1] N → R be a multilinear extension o f a submodular functio n f . Deﬁne a function F ∗ on L N , by F ∗ ( X ) = F ( x ) , wher e x ∈ [0 , 1 ] N is the vector r epr esented by X . Th en F ∗ is submodula r: F ∗ ( X ∪ Y ) + F ∗ ( X ∩ Y ) ≤ F ∗ ( X ) + F ∗ ( Y ) , wher e the union and intersection is interpr eted comp onent- wise. Pr oo f. W e have F ( x ) = E [ f ( ˆ x )] where ˆ x i = 1 ind epend ently with probability x i . An equi valent way to generate ˆ x is to choose any set X ∈ L N representin g x , gen erate uniformly and indepen dently a numb er r i ∈ [0 , 1] f or each i ∈ N , and set ˆ x i = 1 iff r i ∈ X i . Since the measure of X i is x i , ˆ x i = 1 with proba bility e xactly x i . Therefor e, F ∗ ( X ) = F ( x ) = E [ f ( ˆ x )] = E [ f ( { i : r i ∈ X i } )] . Similarly , F ∗ ( Y ) = E [ f ( { i : r i ∈ Y i } )] . This also holds for X ∪ Y and X ∩ Y : since ( X ∪ Y ) i = X i ∪ Y i and ( X ∩ Y ) i = X i ∩ Y i , we get F ∗ ( X ∪ Y ) = E [ f ( { i : r i ∈ X i } ∪ { i : r i ∈ Y i } )] and F ∗ ( X ∩ Y ) = E [ f ( { i : r i ∈ X i } ∩ { i : r i ∈ Y i } )] . Hence, by the submod ularity of f , F ∗ ( X ∪ Y ) + F ∗ ( X ∩ Y ) = E [ f ( { i : r i ∈ X i } ∪ { i : r i ∈ Y i } ) + f ( { i : r i ∈ X i } ∩ { i : r i ∈ Y i } )] ≤ E [ f ( { i : r i ∈ X i } ) + f ( { i : r i ∈ Y i } )] = F ∗ ( X ) + F ∗ ( Y ) . From here, we obtain our main lemma - the a verage of the tw o fractio nal local optima is at least 1 4 O P T . Lemma 3.12. Let O P T = max { F ( x ) : x ∈ P } . Let x b e the o utput of Algorithm 3.6 on polytope P , and z an output of Algorithm 3.6 on polyto pe Q = { y ∈ P : y ≤ 1 − x } , with pa rameter δ as in Algorithm 3.6. Th en with hig h pr o bability , 2 F ( x ) + 2 F ( z ) ≥ O P T − 10 δ n. Pr oo f. Let O P T = F ( x ∗ ) where x ∗ ∈ P . By Lemma 3. 8, the ou tput of the alg orithm x satisﬁes with hig h p robab ility 2 F ( x ) ≥ F ( x ∨ x ∗ ) + F ( x ∧ x ∗ ) − 5 δ n. (1) In the restricted polytope Q = { y ∈ P : y ≤ 1 − x } , consider the po int z ∗ = ( x ∗ − x ) ∨ 0 ∈ Q . Again by Lemma 3. 8, the output of the algorithm z satisﬁes 2 F ( z ) ≥ F ( z ∨ z ∗ ) + F ( z ∧ z ∗ ) − 5 δ n. (2) Now we use a representatio n of vector s by subsets as describe d in Def. 3.10. W e choose X , X ∗ , Z , Z ∗ ∈ L N to represent x , x ∗ , z , z ∗ as fo llows: fo r each i ∈ N , X i = [0 , x i ) , Z i = [ x i , x i + z i ) (n ote that x i + z i ≤ 1 ), X ∗ i = [0 , x ∗ i ) and Z ∗ i = [0 , z ∗ i ) = [0 , max { x ∗ i − x i , 0 } ) . Note that ( X ∩ Z ) i = ∅ for all i ∈ N . 12 Deﬁning F ∗ as in L emma 3.1 1, we have F ∗ ( X ) = F ( x ) , F ∗ ( X ∗ ) = F ( x ∗ ) = O P T , F ∗ ( Z ) = F ( z ) and F ∗ ( Z ∗ ) = F ( z ∗ ) . Using r elations like [0 , x i ) ∪ [0 , x ∗ i ) = [0 , max { x i , x ∗ i } ) , we also get F ∗ ( X ∪ X ∗ ) = F ( x ∨ x ∗ ) and F ∗ ( X ∩X ∗ ) = F ( x ∧ x ∗ ) . Furthermor e, we have ( X ∗ i \X i ) ∪Z i = [ x i , max { x ∗ i , x i + z i } ) = [ x i , x i +max { z ∗ i , z i } ) . This is an interval o f length max { z ∗ i , z i } = ( z ∨ z ∗ ) i and hence F ∗ (( X ∗ \ X ) ∪ Z ) = F ( z ∨ z ∗ ) , where ( X ∗ \ X ) ∪ Z is interpreted compo nent-wise. The pro perty of the ﬁr st local o ptimum (1) can be thus written as 2 F ( x ) ≥ F ∗ ( X ∪ X ∗ ) + F ∗ ( X ∩ X ∗ ) − 5 δ n. The property of th e co mplemen tary local op timum ( 2) can be wr itten a s 2 F ( z ) ≥ F ∗ (( X ∗ \ X ) ∪ Z ) − 5 δ n ( we discarded the nonnegative term F ( z ∧ z ∗ ) which is not used in the follo wing). Th erefore, 2 F ( x ) + 2 F ( z ) ≥ F ∗ ( X ∪ X ∗ ) + F ∗ ( X ∩ X ∗ ) + F ∗ (( X ∗ \ X ) ∪ Z ) − 10 δ n . By Lemma 3.11, F ∗ is submodu lar . Hence we get F ∗ ( X ∩ X ∗ ) + F ∗ (( X ∗ \ X ) ∪ Z ) ≥ F ∗ (( X ∩ X ∗ ) ∪ ( X ∗ \ X ) ∪ Z ) = F ∗ ( X ∗ ∪ Z ) (we d iscarded the intersectio n term) . Finally , u sing the fact that X ∩ Z = ∅ an d again the sub modu larity of F ∗ , we get F ∗ ( X ∪ X ∗ ) + F ∗ ( X ∗ ∪ Z ) ≥ F ∗ (( X ∪ X ∗ ) ∩ ( X ∗ ∪ Z )) = F ∗ ( X ∗ ) (we discarded the union term). T o summarize, 2 F ( x ) + 2 F ( z ) ≥ F ∗ ( X ∪ X ∗ ) + F ∗ ( X ∩ X ∗ ) + F ∗ (( X ∗ \ X ) ∪ Z ) − 10 δ n ≥ F ∗ ( X ∪ X ∗ ) + F ∗ ( X ∗ ∪ Z ) − 10 δ n ≥ F ∗ ( X ∗ ) − 10 δ n = O P T − 10 δ n. Since the parameter δ in Alg orithm 3.6 is chosen as δ = M n a − 1 for some constant a > 3 , we obtain the following. Corollary 3. 13. F or any solva ble d own-mon otone polyto pe P ⊆ [0 , 1] N and mu ltilinear extension of a submodular function F : [0 , 1 ] N → R + , Alg orithm 3.9 ﬁnds with high p r ob ability a solution o f valu e at least 1 4 OP T − O ( M n a − 2 ) for the pr o blem max { F ( x ) : x ∈ P } . W e remark that in many settings of interest, O P T = max { F ( x ) : x ∈ P } ≥ M / p oly ( n ) and thu s we can make the add itiv e error small rela ti ve to the optimum by ch oosing a large en ough . T his leads to a multiplicative (1 / 4 − o (1)) -ap proxim ation. A concrete setting of interest is when P is not too thin in any dimension, as highligh ted by the following lemma wh ich, together with Corollary 3.13, implies Theorem 1.1. Lemma 3.14. Let f : 2 N → R ≥ 0 be a nonn e gative submod ular fun ction with multilinear e xtension F , an d let P ⊆ R n be a solvable down-monotone p olytope satisfying that ther e is a λ ∈ Ω( 1 p oly( n ) ) such tha t for e ach co or dinate i ∈ [ n ] , λ · e i ∈ P . Furthermor e, let O P T = max { F ( x ) | x ∈ P } , and M = max { f ( i ) , f ( N − i ) : i ∈ N } . Then O P T ≥ Ω  M po ly( n )  . Pr oo f. Let i ∈ N . Since F is con cave along any nonnegative d irection and λ e i ∈ P , we have O P T ≥ F ( λ e i ) ≥ λF ( e i ) = λf ( i ) ∀ i ∈ N . (3) Furthermo re, f ( N − i ) ≤ X j ∈ N − i f ( j ) ≤ n λ O P T , (4) where we used (3) for the last inequality . Equatio ns ( 3) and (4) imply the desired results. 13 3.4 Restricted local sear ch: a 0.309-approximation Next, we p resent a modiﬁed local-search alg orithm which is a g eneralizatio n o f the alg orithm f or matroid po lytopes from [50]. W e remar k that this algorithm is in fact simp ler than the 1 4 -appro ximation from the previous section, in that it d oes not req uire a seco nd-stage complem entary local search. Both algorith ms work for any down-mono tone polytop e P . Howev er , o ur an alysis of th e restricted local-search a lgorithm is with respect to th e b est integer solution in the polytope ; we d o not know whether the approx imation g uarantee holds with respect to the best fraction al solution. Algorithm 3.15 . Fix a p arameter t ∈ [0 , 1] . Using Algorithm 3.6, ﬁn d an appr oximate local optimum x in the polyto pe P ∩ [0 , t ] N . Return x . W e show that with th e ch oice of t = 1 2 (3 − √ 5) , th is algor ithm achieves a 1 4 ( − 1 + √ 5) ≃ 0 . 309 -app roxima tion with respect to the optimal integer solution in P . Lemma 3.16. Let x be an outpu t of Algorithm 3.6 on P ∩ [0 , t ] N . Deﬁne w ∈ [0 , 1 ] N by w i = t if x i ≥ t − 1 / n and w i = 1 if x i < t − 1 /n . Let z be any point in P an d let z ′ = z ∧ w . Then w ith high pr obability , 2 F ( x ) ≥ F ( x ∨ z ′ ) + F ( x ∧ z ′ ) − 5 δ n 2 . W e r emark that the above inequality would be imm ediate from Lemma 3 .8, if z ′ ∈ P ∩ [0 , t ] N . Howe ver , z ′ is not necessarily constraine d b y [0 , t ] N . Pr oo f. Consider z ′ = z ∧ w as deﬁned above. By down-monoton icity , z ′ ∈ P . Also, the coordinates whe re z ′ i > t are exactly those where x i < t − 1 /n . So we have x + 1 n ( z ′ − x ) ∈ P ∩ [0 , t ] N . By the stopping rule of Algorithm 3.6, 1 n ( z ′ − x ) · ∇ F ( x ) ≤ 5 δ n. By Lemma 3.2, this implies F ( x ∨ z ′ ) + F ( x ∧ z ′ ) − 2 F ( x ) ≤ ( z ′ − x ) · ∇ F ( x ) ≤ 5 δ n 2 . In the rest of the analysis, we follow [5 0]. Deﬁnition 3.17. F o r x ∈ [0 , 1] N and λ ∈ [0 , 1] , we deﬁne the associated “thr esh old set” as T >λ ( x ) = { i : x i > λ } . Lemma 3.18. Let x ∈ [0 , 1] N . F or any pa rtition N = C ∪ ¯ C , F ( x ) ≥ E [ f (( T >λ ( x ) ∩ C ) ∪ ( T >λ ′ ( x ) ∩ ¯ C ))] wher e λ, λ ′ ∈ [0 , 1] are in depend ently and u niformly random. This ap pears as Lemma A.5 in [50]. W e r emark th at the right-hand side with C = ∅ or C = N gives the Lov ´ asz extension of f and the lemma follows by comp aring the m ultilinear and L ov ´ asz extension. For a no n-trivial partition ( C, ¯ C ) , the lemm a fo llows by two applications o f th is f act. Th e n ext lemma is exactly as in [5 0] fo r th e spec ial case of a matroid polyto pe; we rephrase th e proo f h ere in our more general setting. Lemma 3.1 9. Assume that t ∈ [0 , 1 2 (3 − √ 5)] . Let x be an output of Algorithm 3 .6 on P ∩ [0 , t ] N (with p arameter a ≥ 4 ), and let z = 1 C be any inte ger so lution in P . Then with high pr obability , F ( x ) ≥  t − 1 2 t 2  f ( C ) − O  M n a − 3  . Pr oo f. Deﬁne A = { i : x i ≥ t − 1 / n } and let w = t 1 A + 1 ¯ A , z ′ = z ∧ w as in Lemma 3.16 . Since z = 1 C , we ha ve z ′ = t 1 A ∩ C + 1 C \ A . By Lemma 3.16, we get 2 F ( x ) ≥ F ( x ∨ z ′ ) + F ( x ∧ z ′ ) − 5 δ n 2 . (5) First, let us analyze F ( x ∧ z ′ ) . Since z ′ = t 1 A ∩ C + 1 C \ A and x ∈ [0 , t ] N , we have x ∧ z ′ = x ∧ 1 C . W e apply Lemma 3.18, which states that F ( x ∧ z ′ ) = F ( x ∧ 1 C ) ≥ E [ f ( T >λ ( x ) ∩ C )] . 14 Due to the de ﬁnition of T >λ ( x ) , with probability t − 1 /n we have λ < t − 1 /n and T >λ ( x ) contain s A = { i : x i ≥ t − 1 / n } . Then, f ( T >λ ( x ) ∩ C ) + f ( C \ A ) ≥ f ( C ) by submod ularity . W e conclude that F ( x ∧ z ′ ) ≥  t − 1 n  ( f ( C ) − f ( C \ A )) . (6) Next, let us analyze F ( x ∨ z ′ ) . W e apply Lemma 3.18. W e get F ( x ∨ z ′ ) ≥ E [ f (( T >λ ( x ∨ z ′ ) ∩ C ) ∪ ( T >λ ′ ( x ∨ z ′ ) ∩ ¯ C ))] . The random threshold sets are as follo ws: T >λ ( x ∨ z ′ ) ∩ C = T >λ ( z ′ ) is equal to C with probability t , and equal to C \ A with pro bability 1 − t , by the deﬁn ition of z ′ . T >λ ′ ( x ∨ z ′ ) ∩ ¯ C = T >λ ′ ( x ) ∩ ¯ C is empty with probability 1 − t , because x ∈ [0 , t ] N . (W e ig nore the contribution when T >λ ′ ( x ) ∩ ¯ C 6 = ∅ .) Because λ, λ ′ are indepen dently sampled, we get F ( x ∨ z ′ ) ≥ (1 − t )( tf ( C ) + (1 − t ) f ( C \ A )) . Provided that t ∈ [0 , 1 2 (3 − √ 5)] , we have t ≤ (1 − t ) 2 . Then, we can write F ( x ∨ z ′ ) ≥ t (1 − t ) f ( C ) + tf ( C \ A ) . (7 ) Combining equations (5), (6) and (7), we get 2 F ( x ) ≥ F ( x ∨ z ′ ) + F ( x ∧ z ′ ) − 5 δ n 2 ≥ t (1 − t ) f ( C ) + t f ( C \ A ) + ( t − 1 n )( f ( C ) − f ( C \ A )) − 5 δ n 2 ≥ (2 t − t 2 ) f ( C ) − O  M n a − 3  using δ n 2 = M /n a − 3 . Next, we show ho w the err or term in Lemma 3.19 can b e compared to the optimal v alue. Note that he re we use the fact that in this section, we comp are to 0 / 1 solution s only . The following Lemma is essentially a specialization of Lemma 3.14 to the 0 / 1 ca se. Lemma 3.20 . Suppo se tha t e i ∈ P for each i ∈ N . Let OP T = max { F ( x ) : x ∈ P ∩ { 0 , 1 } N } and M = max i ∈ N { f ( i ) , f ( N − i ) } . Then O P T ≥ 1 n M . Pr oo f. If M = f ( i ) for some i ∈ N , then clearly O P T = max { F ( x ) : x ∈ P ∩ { 0 , 1 } N } ≥ M , becau se F ( e i ) = f ( i ) and e i ∈ P . If M = f ( N − i ) for some i ∈ N , then consider P j 6 = i f ( j ) ≥ f ( N − i ) = M which holds by submodu larity and no nnegativity of f . W e hav e f ( j ) ≥ 1 n M for some j 6 = i . By the above argu ment, OP T ≥ 1 n M . Clearly , if e i / ∈ P , then coordina te x i cannot p articipate in an integer optim um, max { F ( x ) : x ∈ P ∩ { 0 , 1 } N } . W e can remove all such coord inates from the problem. Therefore, we can in fact assum e that e i ∈ P for all i ∈ N . Corollary 3.21. A ssume e i ∈ P for a ll i ∈ N . Then for t = 1 2 (3 − √ 5) , Algorithm 3.1 5 with high p r o bability a 1 4 ( − 1 + √ 5 − o (1)) -a ppr oximation for the pr oblem max { F ( x ) : x ∈ P ∩ { 0 , 1 } N } . 4 Contention r esolution schemes In th is section we discuss conten tion resolution sch emes in more detail and prove our resu lts on the existence o f contention resolution schemes and their application to submod ular maximization problems. 15 4.1 Contention r esolution basics Recall the d eﬁnition, from Section 1, of a ( b, c ) -ba lanced C R scheme π for a po lytope P I . W e ﬁr st prove the claim that a ( b, c ) -balanced C R scheme π can be transformed into a bc -balanced C R scheme π ′ as follows. Let x ∈ P I and A ⊆ N . W e deﬁne π ′ x ( A ) as follows. First each element of A is removed ind ependen tly of the others with prob ability 1 − b to obtain a ran dom set A ′ ⊆ A . W e then set π ′ x ( A ) = π x ( A ′ ) . The key o bservation is tha t if A is a set drawn accordin g to th e distribution induced by R ( x ) , then A ′ has a distribution gi ven b y R ( b x ) . Hence, for any i ∈ N Pr[ i ∈ π ′ x ( R ( x )) | i ∈ R ( x )] = Pr[ i ∈ π ′ x ( R ( x ))] Pr[ i ∈ R ( x )] = Pr[ i ∈ π x ( R ( b x ))] Pr[ i ∈ R ( x )] = b Pr[ i ∈ π x ( R ( b x ))] Pr[ i ∈ R ( b x )] = b Pr[ i ∈ π x ( R ( b x )) | i ∈ R ( b x )] ≥ bc, where the last inequality follows f rom the fact that π is ( b, c ) -balanced. Monotonicity of CR schemes for submodular functio n maximization: Th e inequ ality that relates conte ntion reso- lution to submo dular maximization is given in Th eorem 1.3. A proof of this inequality also appears in [4] fo r monoto ne function s without the pruning procedur e. Befo re pr esenting the p roof, we provide some intuition on why monoto nicity of the C R scheme is needed in the con text of subm odular fu nction m aximization , and w e specif y th e pr uning pr oce- dure η f . It is ea sy to see that if P I has a c -b alanced C R scheme then it imp lies a c -app roxim ation for m aximizing a linear function over P I . I f x is a fraction al solution then its value is P i w i x i , wh ere w i are some (non- negativ e) weights; sin ce each element i is present in the ﬁnal so lution produc ed b y a c -balanced C R scheme with probability at least cx i , by linearity of e xpectation , the expected weig ht o f a solution retur ned by a c -balance d scheme is at least c P i w i x i . More generally , we would like to prove su ch a bound for any sub modu lar function f via F . Howe ver , this is no longer o bvious since elements do n ot app ear inde penden tly in the rounding scheme; recall th at F ( x ) is the expected value o f f on a set pro duced by indepen dently including each i with probability x i . Mono tonicity is the proper ty that is useful in this context, because elements of smaller sets contrib ute more to a submodular function than elements of larger sets. T o prove Theo rem 1.3, we ﬁrst introd uce th e claimed p runing function η f . T o pru ne a set I via the prun ing function η f , an arb itrary o rdering o f th e elements o f N is ﬁxed: for notational simp licity let N = { 1 , . . . , n } which giv es a natural o rdering . Starting with J = ∅ the ﬁnal set J = η f ( I ) —which we called the pru ned version of I —is constructed by g oing throu gh all elemen ts of I in th e order indu ced by N . When co nsidering an elem ent i , J is replaced by J + i if f ( J + i ) − f ( J ) > 0 . Pr oo f of Theor em 1.3. L et R = R ( x ) and I = π x ( R ) , a nd let J = η f ( I ) if f is non -mono tone an d J = I otherwise. Hence, in both cases, J is the set returned by the suggested round ing p rocedu re. Assume that N = { 1 , . . . , n } is the same orderin g o f the elemen ts as used in the prunin g o peration (in case no prunin g was applied, any order is ﬁn e). The main pr operty we get by pruning is the f ollowing. Notice that this pro perty trivially holds wh en f is mono tone. f J ∩ [ i − 1] ( i ) ≥ 0 ∀ i ∈ J . (8) Furthermo re, for each i ∈ I f J ∩ [ i − 1] ( i ) > 0 ⇒ i ∈ J. (9) Again, notice that this proper ty h olds tri vially in the mono tone ca se in which we have J = I . The main step that we will prove is that for any ﬁxed i ∈ { 1 , . . . , n } , E [ f ( J ∩ [ i ]) − f ( J ∩ [ i − 1])] ≥ c E [ f ( R ∩ [ i ]) − f ( R ∩ [ i − 1])] . (10) W e highlight that there are tw o sources of randomn ess over which the e xpectation is taken on the left-hand side of the above inequality : on e source is the randomn ess in choosing the set R , and the other source is the potential rand omness of the CR schem e used to ob tain the set I fro m R , which is later de terministically prun ed to get J . The theorem then follows from (10) s ince E [ f ( J )] = f ( ∅ ) + n X i =1 E [ f ( J ∩ [ i ]) − f ( J ∩ [ i − 1])] ≥ f ( ∅ ) + c n X i =1 E [ f ( R ∩ [ i ]) − f ( R ∩ [ i − 1])] ≥ c E [ f ( R )] . 16 Hence, it remains to prove (10). Conside r ﬁrst the non-mo notone case. Here we h av e E [ f ( J ∩ [ i ]) − f ( J ∩ [ i − 1])] = E [ 1 i ∈ J f J ∩ [ i − 1] ( i )] = P r[ i ∈ R ] · E [ 1 i ∈ J f J ∩ [ i − 1] ( i ) | i ∈ R ] (8) = Pr[ i ∈ R ] · E [ 1 i ∈ J max { 0 , f J ∩ [ i − 1] ( i ) } | i ∈ R ] (9) = Pr[ i ∈ R ] · E [ 1 i ∈ I max { 0 , f J ∩ [ i − 1] ( i ) } | i ∈ R ] ≥ P r[ i ∈ R ] · E [ 1 i ∈ I max { 0 , f R ∩ [ i − 1] ( i ) } | i ∈ R ] (since f is submod ular) = P r[ i ∈ R ] · E [ E [ 1 i ∈ I max { 0 , f R ∩ [ i − 1] ( i ) } | R ] | i ∈ R ] = P r[ i ∈ R ] · E [ E [ 1 i ∈ I | R ] max { 0 , f R ∩ [ i − 1] ( i ) } | i ∈ R ] . On the pro duct space associated with the d istribution o f R condition ed on i ∈ R , bo th of the terms E [ 1 i ∈ I | R ] and max { 0 , f R ∩ [ i − 1] ( i ) } are non-inc reasing fu nctions, b ecause of the mon otonicity of th e C R scheme used to obtain I from R and f being submo dular, respecti vely . Notice that the rando mness in both terms E [ 1 i ∈ I | R ] and max { 0 , f R ∩ [ i − 1] ( i ) } stems o nly from the ran dom set R , and not fro m the potential rand omness of the CR scheme. Hence, by the FKG inequality we obtain Pr[ i ∈ R ] · E [ E [ 1 i ∈ I | R ] max { 0 , f R ∩ [ i − 1] ( i ) } | i ∈ R ] ≥ Pr[ i ∈ R ] · E [ 1 i ∈ I | i ∈ R ] · E [max { 0 , f R ∩ [ i − 1] ( i ) } | i ∈ R ] = Pr[ i ∈ R ] · Pr[ i ∈ I | i ∈ R ] · E [max { 0 , f R ∩ [ i − 1] ( i ) } | i ∈ R ] ≥ c P r[ i ∈ R ] · E [max { 0 , f R ∩ [ i − 1] ( i ) } | i ∈ R ] ≥ c P r[ i ∈ R ] · E [ f R ∩ [ i − 1] ( i ) | i ∈ R ] = c P r[ i ∈ R ] · E [ f R ∩ [ i − 1] ( i )] = c E [ f ( R ∩ [ i ]) − f ( R ∩ [ i − 1 ])] , where in the second to last equality we use again th e p roperty th at f R ∩ [ i − 1] ( i ) is ind ependen t of i ∈ R . Hence, this shows ( 10) as desired, and completes the proo f. The following subsectio n on strict CR schemes discusses an alternate way of rou nding th at does not rely on p runing and is ob livious to th e u nderly ing subm odular fu nction. As we h ighlight below , such a procedure is useful wh en th e value of se veral submodular function s should approxima tely b e preserved , simultaneously . Howe ver , since we do no t rely on this alternate proced ure later , this part can safely be skipped. Strict co ntention resolution schemes An a lternative way to round in the context of non- monoto ne subm odular function s, that does not rely on pru ning, c an b e o btained b y using a strong er no tion of C R schemes. Mor e pr ecisely , we say that a ( b, c ) -b alanced C R scheme π for P I is strict , if it satisﬁes the second condition of a C R scheme with equality , i.e., Pr[ i ∈ π x ( R ( x ))] = c . W e ha ve the fo llowing ( the proof can be foun d in Ap pendix B). Theorem 4. 1. Let f : 2 N → R + be a n on-negative submod ular functio n with multilinear r elaxatio n F , and x be a point in P I , a conve x r ela xation for I ⊆ 2 N . Let π be a mo noton e and strict ( b, c ) -b alanced C R scheme for P I , and let I = π x ( R ( x )) . Then E [ f ( I )] ≥ c F ( x ) . The advantage of using a strict C R scheme comp ared to applying the p runin g step is that th is version of rou nding is oblivious to the underlying submodular fun ction f . This could potentially b e useful in settings where one is interested in simultaneously maximizin g more than one submodular function. Assum e for example that x is a point such that F 1 ( x ) an d F 2 ( x ) h av e simu ltaneously hig h values, where F 1 and F 2 are the m ultilinear relaxations of two submod ular function s f 1 and f 2 . Then using a rou nding that is ob livious to th e und erlying submodu lar fu nction leads to a randomly round ed s et I satisfying E [ f 1 ( I )] ≥ cF 1 ( x ) and E [ f 2 ( I )] ≥ cF 2 ( x ) . Any monotone but not necessarily strict ( b, c ) -balanced C R scheme π can be transf ormed into a mo noton e ( b, c ) - balanced C R scheme th at is arbitrarily close to being strict as follo ws. For each element i ∈ N , o ne can estimate the p robab ility c ′ i = Pr[ i ∈ π x ( R ( x )) | i ∈ I ] ≥ c via Monte- Carlo sampling within a polyno mially small er ror 17 (assuming that c is a co nstant). Then we can modify the C R scheme by removing fr om its output I , eleme nt i ∈ I with probab ility 1 − c/c ′ i . The resulting sch eme is ar bitrarily close to being strict and can be u sed in place of a strict scheme in Th eorem 4.1 with a weaker guarantee; in applications to approx imation, the ratio is affected in the lower -order terms. W e om it further details. Combining C R schemes: Next, we discuss how to combine co ntention reso lution schem es fo r d ifferent constraints. W e co nsider a constraint I = ∩ h i =1 I i and its polyhedral rela xation P I = ∩ i P I i , such that P I i has a monotone ( b, c i ) - balanced C R scheme π i . W e pro duce a contentio n resolution scheme π for I which works with respe ct to the natur al combinatio n of constraint relaxations — an intersection of the respective polyto pes P I i . This ensur es that the relaxed problem is still tractable and we can apply our optimization framework. In case som e elements D ⊆ N are not p art of the constraint I i , we assume w ithout loss of g enerality that π i never removes elemen ts in D , i.e., π i x ( A ) ∩ D = A ∩ D for any x ∈ bP I i and A ⊆ N . The combined contention resolution scheme π for P I is deﬁned by π x ( A ) = \ i π i x ( A ) fo r A ⊆ N , x ∈ bP I . A straigh tforward union bound would state that the com bined sch eme π is ( b, 1 − P i (1 − c i )) -balanced for P I . Using the FKG inequality , we obtain a stronger r esult in this setting , namely a ( b, Q i c i ) -balanced scheme. Moreover , if each constraint admits a ( b, c ) -balan ced scheme a nd each element participates in at most k co nstraints, then we obtain a ( b, c k ) -balanced scheme. This is the statemen t o f Lemma 1.6 which we prove here u sing the combin ed scheme π deﬁned abov e. Pr oo f of Lemma 1.6. Let us consider the ≤ k con straints that e lement i participates in. For simplicity we assume k = 2 ; the general statement follows b y induction . For notation al con venience we deﬁne R = R ( x ) , I 1 = π 1 x ( R ) and I 2 = π 2 x ( R ) . Conditioned on R , the choices of I 1 , I 2 are independ ent, wh ich means that Pr[ i ∈ I 1 ∩ I 2 | R ] = Pr[ i ∈ I 1 & i ∈ I 2 | R ] = Pr[ i ∈ I 1 | R ] Pr[ i ∈ I 2 | R ] . T aking an e xpectation over R co nditioned on i ∈ R , we get Pr[ i ∈ I 1 ∩ I 2 | i ∈ R ] = E R [Pr[ i ∈ I 1 ∩ I 2 | R ] | i ∈ R ] = E R [Pr[ i ∈ I 1 | R ] Pr[ i ∈ I 2 | R ] | i ∈ R ] . Both Pr[ i ∈ I 1 | R ] an d Pr[ i ∈ I 2 | R ] are no n-incre asing fu nctions of R on the pro duct space of sets con taining i , so by the FKG inequality , E R [Pr[ i ∈ I 1 | R ] Pr[ i ∈ I 2 | R ] | i ∈ R ] ≥ E R [Pr[ i ∈ I 1 | R ] | i ∈ R ] · E R [Pr[ i ∈ I 2 | R ] | i ∈ R ] . Since these expectations are simply probabilities conditioned on i ∈ R , we conclude : Pr[ i ∈ I 1 ∩ I 2 | i ∈ R ] ≥ Pr[ i ∈ I 1 | i ∈ R ] P r[ i ∈ I 2 | i ∈ R ] . Monoton icity of the abov e sche me is also easily implied: consider j ∈ T 1 ⊂ T 2 ⊆ N , then Pr[ j ∈ I | R = T 1 ] = Y i Pr[ j ∈ I i | R = T 1 ] ≥ Y i Pr[ j ∈ I i | R = T 2 ] = P r[ j ∈ I | R = T 2 ] . where the ineq uality f ollows from the fact that each of the schemes is mo notone . Th e polynomial time implementabil- ity of the compo sed s cheme follows e asily from the polyno mial time implementability of π 1 and π 2 . 4.2 Obtaining C R schemes via distribu tions of deterministic C R schemes W e now describe a g eneral way to obtain C R schemes r elying on an LP appr oach. More precisely , we will observe that any C R scheme c an b e in terpreted as a distribution over deterministic C R schemes. Explo iting this ob servation, we formu late an exponen tial-sized LP whose optimal solution corr esponds to an op timal C R scheme. The separation problem of its d ual th en g iv es a natural ch aracterization fo r th e existence of stron g CR schemes, which can be m ade 18 algorithm ic in som e interesting cases includ ing m atroid con straints, as we show in Sectio n 4.4. Furtherm ore, in Section 4.3, we will use this point of view to draw a connection to a recently in troduce d concept, known as correlation gap. Recall the formal deﬁnition of C R schemes gi ven in Deﬁnition 1.2, in particula r the d ifferences b etween o blivious, deterministic and g eneral (ran domized ) sch emes. First, we no te that the simplest C R schemes are the o blivious ones. An oblivious scheme do es not dep end on x an d is d eterministic; h ence it is essentially a single mapping π : 2 N → I that given A ⊆ N return s a set π ( A ) such that π ( A ) ⊆ A and π ( A ) ∈ I . Several alteratio n based schemes are oblivious — see [5, 14] fo r so me exam ples. A typical o blivious scheme ﬁxes an order ing of the elemen ts of N (that depend s o n the combina torial p roperties of I ); it starts with an empty set A ′ , and consider s the elements of A accordin g to the ﬁxed order and adds the cu rrent e lement i to the set A ′ if A ′ ∪ { i } ∈ I , oth erwise it discards i . Finally it outputs A ′ . These greedy ord ering based insertion schemes are easily seen to be mon otone. A deterministic C R scheme is more gener al than an obli vious scheme in that the outpu t can depend on x ; in other words, for each x ∈ P I , π x is a mapping from 2 N to I . The ad vantage or need for such a d epend ence is dem onstrated by matroid po lytopes. L et P ( M ) be the con vex hull of th e independent sets of a matroid M ; ob livious schemes canno t give a c -balanc ed C R scheme fo r any con stant c . Howe ver , we ca n show that fo r any b ∈ [0 , 1] a go od de terministic CR scheme exists: for any x ∈ P M , the re is an ordering σ x that can b e efﬁciently computed fr om x such that a greedy in sertion scheme based on the order ing σ x giv es a ( b, 1 − b ) -balanced scheme. Such a scheme for b = 1 / 2 is implicitly presen t in [11], howe ver for completeness, we gi ve the details of our scheme in S ection 4.4. The algorithm in [ 10] for geometric packing pr oblems was re interpreted as a deterministic C R scheme f ollowing our work; it is also b ased on computing an o rderin g that d epends o n x followed by a gre edy inser tion pro cedure via the co mputed o rdering (see also more recent work [19]). Such ordering based deterministic schemes are easily seen to be monoton e. In contrast to deter ministic schemes, g eneral (ran domized ) C R schemes are such that π x ( A ) is a rando m feasible subset of A . Randomiza tion is necessary to obtain an op timal result even when considering contention for a sing le item [ 22, 2 3]. F or the time b eing, we do not req uire th e C R schemes to be monotone; this is a point we discuss later . A non-o blivious ( b, c ) -balanced C R sch eme π , determin istic or randomized , can depend on x , and henc e it is co nv enient to vie w it is a collection of separate schemes, on e f or each x ∈ bP I . They ar e only tied toge ther by the un iform guaran tee c . In the fo llowing we will ﬁx a particular x and f ocus on ﬁnd ing the b est schem e π x for it. As we alr eady discussed, if π is determin istic, then π x is a mappin g from 2 N to I . W e observe that a r andomiz ed sch eme π x is a distribution over deterministic schemes; n ote th at here we are ignorin g computational issues as well as monotonicity . W e formalize this now . Call a map ping φ from 2 N to I valid if φ ( A ) ⊆ A ∀ A ⊆ N . Let Φ ∗ be the family of all valid map pings f rom 2 N to I . Any probability distrib ution ( λ φ ) , φ ∈ Φ ∗ induces a randomized schem e π x as follows. For a set A , the algorithm π x ﬁrst picks φ ∈ Φ ∗ accordin g to the given prob ability d istribution and then o utputs φ ( A ) . Con versely , for e very ra ndomize d scheme π x , th ere i s an associated prob ability distribution ( λ φ ) , φ ∈ Φ ∗ 10 . Based on the preced ing observation, one can write an LP to expr ess the problem of ﬁndin g a C R scheme that is ( b, c ) -balanced for x with a v alue of c as high as possible. More precisely , for e ach φ ∈ Φ ∗ , we deﬁne q i,φ = P r[ i ∈ φ ( R )] , where, as usual, R := R ( x ) is obtained by includin g e ach j ∈ N in R with p robability x j , independ ently of the oth er elements. Thus, for a g iv en distribution ( λ φ ) φ ∈ Φ ∗ , the pro bability that th e corresp onding C R sche me π x returns a set π x ( R ) containing i , is given by P φ ∈ Φ ∗ q i,φ λ φ . Hence, the problem of ﬁndin g the distrib ution ( λ φ ) π ∈ Φ ∗ that leads to a ( b, c ) - balanced C R scheme fo r x with c as high as possible can be fo rmulated as the following line ar p rogram ( LP1), with correspo nding dual ( DP1). (LP1) max c s.t. P φ ∈ Φ ∗ q i,φ λ φ ≥ x i c ∀ i ∈ N P φ ∈ Φ ∗ λ φ = 1 λ φ ≥ 0 ∀ φ ∈ Φ ∗ (DP1) min µ s.t. P i ∈ N q i,φ y i ≤ µ ∀ φ ∈ Φ ∗ P i ∈ N x i y i = 1 y i ≥ 0 ∀ i ∈ N 10 Let k be an upper bound on the number of random bits used by π x . For any ﬁxed string r of k random bits, let φ r be the val id mapping from 2 N to I gene rated by the algorithm π x with random bits set to r . The distrib ution where for each r the probability assigned to φ r is 1 / 2 k is the desired one. 19 In gen eral we may also be intere sted in a restricted set of map pings Φ ⊆ Φ ∗ . In the above L P we can replace Φ ∗ by Φ to obtain the best c that can be achiev ed by taking prob ability distributions over valid map pings in Φ . Let c ( x , Φ) be th e o ptimum value of the LP for a gi ven x and a set Φ ⊆ Φ ∗ . It is easy to see that c ( x , Φ) ≤ c ( x , Φ ∗ ) f or any Φ . From the earlier discussion, c ( x , Φ ∗ ) is the best scheme for x . W e sum marize the discussion so far by the following. Proposition 4.2. Th er e e xists a ( b, c ) -bala nced C R scheme for P I iff inf x ∈ bP I c ( x , Φ ∗ ) ≥ c . Proving the ex istence of a ( b, c ) -ba lanced C R scheme: T o show that P I has a ( b , c ) -balan ced C R scheme w e nee d to show that c ( x , Φ ∗ ) ≥ c for all x ∈ bP I . By LP d uality this is equi valent to sho wing that the optimum value of the dual (DP1) is at least c for all x . W e ﬁrst reform ulate the dual in a co n venient fo rm s o that proving a lower b ound c o n the dual optimum reduces to a more intuitive question. W e will th en address the issue o f ef ﬁciently co nstructing a C R scheme that nearly matches the lower bound. Below we will use R to d enote a random set obtain ed by p icking each i ∈ N indep endently with proba bility x i and use prob abilities and e xpectation s with respect to this rando m pro cess. Th e optimu m value of the du al can be rewritten as: min y ≥ 0 max φ ∈ Φ ∗ P i q i,φ y i P i ∈ N x i y i = min y ≥ 0 max φ ∈ Φ ∗ P i y i Pr[ i ∈ φ ( R )] P i ∈ N x i y i = min y ≥ 0 max φ ∈ Φ ∗ E R h P i ∈ φ ( R ) y i i P i ∈ N x i y i For any ﬁxed weight vector y ≥ 0 we cla im that max φ ∈ Φ ∗ E R   X i ∈ φ ( R ) y i   = E R " max S ⊆ R,S ∈I X i ∈ S y i # , which fo llows by co nsidering the speciﬁc map ping φ ∈ Φ ∗ that f or each A ⊆ N sets φ ( A ) = max A ′ ⊆ A,A ′ ∈I y ( A ) . Thus, the dual optimum value is min y ≥ 0 E R  max S ⊆ R,S ∈I P i ∈ S y i  P i ∈ N x i y i . (11) The above expression can be explained as an “integrality gap ” o f P I for a speciﬁc r oundin g strategy; h ere the problem o f in terest is to ﬁnd a maximum weigh t ind epend ent set in I . The vector y co rrespond s to weig hts on N . The vector x corr esponds to a frac tional s olution in bP I (it is helpful here to think of b = 1 ). Thu s P i ∈ N x i y i is the value of the f ractional solution . T he num erator is the expected value of a ma ximum weight ind ependen t set in R . Since we are m inimizing over y , th e r atio is th e worst case gap between the value of a n in tegral feasible solutio n (ob tained via a speciﬁc round ing) a nd a fractional solution. Thus, to pr ove the existence of a ( b, c ) -b alanced C R scheme it is suf ﬁcient (and necessary) to prove that for all y ≥ 0 and x ∈ b P I E R ( x )  max S ⊆ R,S ∈I P i ∈ S y i  P i ∈ N x i y i ≥ c. Constructing C R schemes via the ellipsoid algorithm: W e now discuss ho w to efﬁciently com pute th e best C R scheme for a given x by solvin g (LP1) via the dual (DP1). W e observe that c ( x , Φ ∗ ) , the best bou nd fo r a given x , could b e smaller tha n the bou nd c . It sho uld not be surprising that the separation oracle fo r the dua l (DP1) is related to the p receding characterization . The separation oracle for ( DP1) is the following: given µ and we ight vector y , no rmalized such that P i x i y i = 1 , chec k whether there is any φ ∈ Φ ∗ such that P i ∈ N q i,φ y i > µ and if so output a separating hyperplane. T o see wh ether there is a violated constraint, it suf ﬁces to e v aluate max φ ∈ Φ ∗ q i,φ y i and compar e it with µ . Follo wing the previous discussion, this expr ession is equal to E R ( x )  max S ⊆ R,S ∈I P i ∈ S y i  . One can accura tely estimate this quantity as follows. First, we sam ple a ran dom set R using marginals given b y x . Then we ﬁnd a max imum y -weight subset of R th at is contain ed in I . This gives an u nbiased estimator , and to get a h igh-acc uracy estimate we repeat the process sufﬁciently many times and take the a verag e value. Thu s, the algorithm ic pr oblem needed for the separation oracle is the m aximum weight independen t set pr oblem for I : given 20 weights y on N and a A ⊆ N outp ut a maximum weight sub set of A in I . The sam pling creates an additi ve er ror ǫ in estimating E R ( x )  max S ⊆ R,S ∈I P i ∈ S y i  which results in a correspo nding loss in ﬁnding the optim um solutio n value µ ∗ to (D P1). T o implem ent the ellipsoid algor ithm we also need to ﬁnd a separating hyper plane if th ere is a v iolated constraint. A n atural strategy would be to outp ut the hyper plane cor respond ing to the violating co nstraint found while ev a luating max φ ∈ Φ ∗ q i,φ y i . Howe ver , we do not necessarily hav e the exact coefﬁcients q i,φ for the constraint since we use ran dom samplin g. W e d escribe in Section C o f the appendix the technica l d etails in implemen ting the ellipsoid algo rithm with sufﬁciently a ccurate estimates obta ined from sampling. For now assume we can ﬁnd a separating hyp erplane correspond ing to the most violated constraint. T he ellipsoid algo rithm can then b e used to ﬁnd a polynom ial nu mber of dual co nstraints that ce rtify that the d ual optimum is at least µ ∗ − ε where µ ∗ is the actual dual optimum value. By strong d uality µ ∗ = c ( x , Φ ∗ ) . W e then solv e th e p rimal (LP1) by restricting it to the v ariables that corre spond to the dual constraints found by the ellipsoid algor ithm. This gives a prim al feasible solution of value c ( x , Φ ∗ ) − ε and this solution is the desired C R scheme. W e ob serve that the primal can be solved efﬁciently sinc e the num ber of variables and constrain ts is po lynomial; here too we do not have the precise co efﬁcients q i,φ but we can u se the esimates that come fro m th e dual — see Section C. T o sum marize, an alg orithm for ﬁnd ing a maximum weight indep endent set in I , tog ether with sampling and the ellipsoid algor ithm, ca n be used to efﬁciently ﬁnd a ( b, c ( x , Φ ∗ ) − ε ) -balanced C R sch eme where ε is an error toler ance; the running time d epends polynomially on the input size and 1 /ε . The proof can be easily adapted to sho w that an α -approximatio n for the max -weight independen t set problem giv es a α · c ( x , Φ ∗ ) − ε C R schem e. Monotonicity: The discussion so far did not c onsider the issue of monotonicity . On e way to adapt t he abov e appr oach to monotone sch emes is to deﬁne Φ to be th e family of all d eterministic monotone C R scheme s a nd solve (LP1) re- stricted to Φ . A deterministic scheme φ is m onoton e if it h as the pro perty that i ∈ φ ( A ) implies that i ∈ φ ( A ′ ) for all A ′ ⊂ A . Distributions of deterministic monoto ne schemes certainly yield a monotone C R scheme. Interestingly , it is not true that all mono tone ra ndomize d C R schemes can b e obtained as distributions of d eterministic ones. Now the question is whether we can solve (LP1 ) r estricted to monoton e deterministic schem es. In g eneral th is is a no n-trivial problem . Howe ver , the ellipsoid-b ased algorithm to compute c ( x , Φ ∗ ) that we described above gi ves the following importan t pr operty . I n each iteration of the ellipsoid algorithm, the separation oracle uses a maximum-weig ht inde- penden t set algorithm for I to ﬁnd a v iolating constra int; this co nstraint corresp onds to a determin istic scheme φ that is obtained by specializing the algorithm to the given weight vector y . Theref ore, if the maximum-weig ht in depen - dent set algo rithm is monoton e, th en all the constraints g enerated in the ellipsoid algo rithm co rrespon d to monotone schemes. Since we solve the primal (LP1) only for th e schemes generated by the separ ation oracle f or the dual ( DP1), it follo ws that there is an op timum so lution to (LP1) that is a distribution over mon otone scheme s! In such a case c ( x , Φ ∗ ) = c ( x , Φ) and there is no loss in using mo noton e schemes. For matroids the g reedy algorithm to ﬁnd a maximum weig ht indep endent set is a mo notone algorith m. T hus, fo r matroid s, the above approach of solv ing (DP1) and (LP1) can be used to o btain a close to optimal m onoto ne ( b, c ) -balan ced C R sch eme. It remain s to determine the value of the optimal c and we analyze it in Section 4.4. It may be the case that th ere is no monotone max imum weight independent set algorithm fo r som e g iv en I , say the intersectio n o f two matroids. In that case we can use an approx imate montone a lgorithm instead. W e summarize the abov e discussions in the follo wing theorem. Theorem 4.3. There is a ( b, c ) -ba lanced C R scheme for P I iff E R ( x )  max S ⊆ R,S ∈I P i ∈ S y i  ≥ c P i y i x i for all x ∈ b P I and y ≥ 0 . Moreo ver , if there is a polyno mial-time deterministic algorithm to ﬁnd a maximum weight indepen dent set in I , then fo r a ny b and ε > 0 , ther e is a rando mized efﬁciently implemen table ( b, c ∗ − ε ) - balanc ed C R scheme for P I wher e c ∗ is th e smallest va lue o f c such that there is a ( b, c ) -balan ced C R scheme for P I ; the running time is polynomial in th e input siz e and 1 /ε . In addition, if the ma ximum-weight independen t set algo rithm is monoton e, the r esulting C R sc heme is monoton e. Before leveraging the above theorem to d esign close to optimal CR schemes for matroid s, we h ighligh t an inter- esting co nnection between CR schemes and a concept known as correlation gap . Th is con nection is a further insight that we gain through the linear program s (LP1) an d (DP1). 21 4.3 Connection to corr elation gap In this section we highligh t a close connection between C R sche mes and a concep t k nown as corr e lation gap [2]. T he correlation gap is a fun ction on set f unction s that measures how muc h the expected value of a set function with respect to some random inp ut can v ary , if o nly the marginal prob abilities of the input are ﬁx ed. W e ﬁrst sho w h ow on e can naturally e xtend this notion to sets I ⊆ 2 N . Then, by explo iting the dual LP formulation o f C R sche mes (DP1 ), w e present a close r elationship of the n otion of correlation g ap, interpreted in terms o f constraints, and the existence of strong C R schemes. Deﬁnition 4.4. F or a set fun ction f : 2 N → R + , the correlation gap is deﬁned as κ ( f ) = inf x ∈ [0 , 1] N E [ f ( R ( x ))] f + ( x ) , wher e R ( x ) is a r andom set indepen dently containing each elemen t i with pr obability x i , and f + ( x ) = max { X S α S f ( S ) : X S α S 1 S = x , X S α S = 1 , α S ≥ 0 } is the ma ximum possible expectation of f over distributions with expectation x . Furthermor e, fo r a c lass of fu nctions C , the corr elation gap is deﬁned by κ ( C ) = inf f ∈C κ ( f ) . In othe r words, the correlation g ap is th e worst-case r atio b etween the multilinear extension F ( x ) = E [ f ( R ( x ))] and the concave clo sure f + ( x ) . W e remark that we deﬁne the cor relation gap as a number κ ∈ [0 , 1] , to be in line with the pa rameter c in our n otion o f a ( b, c ) -ba lanced C R scheme ( the higher the better). T he deﬁnition in [2] uses the in verse r atio. The relationship between C R schemes and correlation gap arises as follows. Deﬁnition 4.5. F or I ⊆ 2 N , we deﬁn e the corr elation gap as κ ( I ) = inf x ∈ P I , y ≥ 0 1 P i x i y i E [max S ⊆ R,S ∈I P i ∈ S y i ] , wher e R = R ( x ) con tains element i in depen dently with p r ob ability x i . The reason we call this q uantity a correlation gap (co nsidering Deﬁnitio n 4.4), is that this q uantity is e qual to the correlation gap of the weighted rank function corresponding to I (see Lemma 4.7 below). Theorem 4.6. The correlation gap of I is equal to the maximum c such th at I admits a c -b alanced C R s cheme. Pr oo f. The co rrelation gap of I is eq ual to the op timum value of (DP1). By LP du ality , this is eq ual to the optimu m of the primal (LP1), which is the best value of c f or which there is a c -balanced C R s cheme. The following lemma shows a close con nection between th e correlation gap of a s olution set I and the correlation gap of the respecti ve rank function. Mor e p recisely , the co rrelation gap of I corresp onds to the w orst (i.e. smallest) correlation gap of the respective rank functio n o ver all weight vectors. Lemma 4 .7. F or I ⊆ 2 N and weight vecto r y ≥ 0 , let r y ( R ) = max S ⊆ R,S ∈I P i ∈ S y i denote the associa ted weighted rank function. Then κ ( I ) = inf y ≥ 0 κ ( r y ) . Pr oo f. Using the notation r y ( R ) for the weig hted rank fun ction with weig hts y , the corr elation gap of I can be rewritten as κ ( I ) = inf x ∈ P I , y ≥ 0 E [ r y ( R ( x ))] P i x i y i , where R ( x ) contains elements indep endently with prob abilities x i . W e ﬁrst obser ve that for any x ∈ P I , we h ave r + y ( x ) = P i x i y i . Hence, let x ∈ P I , an d consider a conve x combination x = P S ∈I α S 1 S , P α S = 1 , α S ≥ 0 with r + y ( x ) = P S ∈I α S y ( S ) . Sinc e the weigh ted rank fun ction of a f easible set S ∈ I is simply its weight we o btain r + y ( x ) = X S ∈I α S y ( S ) = y · X S ∈I α S 1 S = y · x = X i x i y i , as claimed. Therefor e, κ ( I ) = inf x ∈ P I , y ≥ 0 E [ r y ( R ( x ))] P i x i y i = inf x ∈ P I , y ≥ 0 E [ r y ( R ( x ))] r + y ( x ) . 22 T o prove the claim it remain s to sho w that inf x ∈ P I , y ≥ 0 E [ r y ( R ( x ))] r + y ( x ) = inf x ∈ [0 , 1] N , y ≥ 0 E [ r y ( R ( x ))] r + y ( x ) . (12) Let y ≥ 0 . W e will prove (12) by sho wing that for any poin t x ∈ [0 , 1] N there is a point x ′ ∈ P I with x ′ ≤ x (coord inate-wise), and satisfying r + y ( x ′ ) ≥ r + y ( x ) . Since r y is monoton e, we then o btain E [ r y ( R ( x ))] /r + y ( x ) ≥ E [ r y ( R ( x ))] /r + y ( x ′ ) , showing that the in ﬁnum over x o n the right-hand si de of (12) can indeed be restricted to the polytop e P I . Let x = P S ⊆ N α S 1 S , P S ⊆ N α S = 1 , α S ≥ 0 be a con ve x combinatio n of x suc h th at r + y ( x ) = P S ⊆ N α S r y ( S ) . For every S ⊆ N , let I ( S ) ⊆ S be a max imum weig ht in depend ent set, hence r y ( S ) = y ( I ( S )) . The point x ′ = P S ⊆ N α S 1 I ( S ) clearly satisﬁes x ′ ≤ x , and fu rthermo re r + y ( x ′ ) ≥ X S ∈I X W ⊆ N ,I ( W )= S α W ! r y ( S ) = X S ⊆ N α S r y ( S ) = r + y ( x ) . 4.4 Contention r esolution for matr oids In this section we prove the following theorem on C R schemes for m atroids. Theorem 4. 8. F or any matr oid M = ( N , I ) o n n elements there exists a  b, 1 − (1 − b n ) n b  -balan ced C R scheme for the polytope P ( M ) . W e later address monoto nicity of the scheme an d constru ctiv e aspects. T o p rove Theorem 4.8 we r ely on the characterizatio n formalize d in T heorem 4 .3. It sufﬁces to prove fo r x ∈ b · P I and any non-negati ve weight vector y ≥ 0 that E R  max S ⊆ R,S ∈I P i ∈ S y i  ≥ c P i ∈ N x i y i , with c = 1 − (1 − b n ) n b where R contain s each i ∈ N indepen dently with prob ability x i and x ∈ b · P I . For a giv en weight vector y ≥ 0 on N a nd a set S ⊆ N let r y ( S ) deno te the weight of a m aximum weigh t ind epend ent set co ntained in S ; in o ther words r y is th e weig hted ran k fun ction of the matroid M . Restating, it remains to prove E [ r y ( R )] ≥ 1 − (1 − b n ) n b X i ∈ N y i x i , (13) It is well-known that a simple greed y algorithm can be used to com pute r y ( S ) (in fact an ind epende nt set S ′ ⊆ S of maximum weight with respect to y i ): Start with S ′ = ∅ , consider the elements o f S in non-increasing ord er of their weight y i and add the current element i to S ′ if S ′ + i is ind epend ent, oth erwise discard i . T o show (13), whic h is a general property of weighted matroid ran k f unctions, we prove a m ore gen eral result that h olds for any n on-negative mon otone submodu lar function . The main in gredient for th is is a lo wer bound on the multilinear extension, which is stated in Lem ma 4.10. A sligh tly weaker for m of Lemma 4 .10, wh ich we state as Lemma 4.9, will b e presented ﬁrst, du e to its consice proof. The proo f o f Lemma 4.10 is deferr ed to the appendix. Both lemmas can b e seen as an extension of the pr operty that the co rrelation gap for m onoto ne s ubmod ular functions is 1 − 1 /e [6]. Lemma 4.9 . If f : 2 N → R + is a monoton e submod ular functio n, F : [0 , 1] N → R + its mu ltilinear extension, a nd f + : [0 , 1] N → R + its concave closur e, then for any b ∈ [0 , 1] and p ∈ [0 , 1] N , F ( b · p ) ≥ (1 − e − b ) f + ( p ) . Pr oo f. W e use another extension of a m onoton e su bmodu lar function, deﬁned in [6]: f ∗ ( p ) = min S f ( S ) + X i p i f S ( i ) ! . 23 It is shown in [6 ] that f ∗ ( p ) ≥ f + ( p ) for all p ∈ [0 , 1] N . Consider the fu nction φ ( t ) = F ( t p ) fo r t ∈ [0 , 1 ] , i.e. the multilinear extension on the line segmen t between 0 an d p . W e pr ove that φ ( t ) satisﬁes a differential equation similar to the analysis of the co ntinuou s gre edy algorithm [ 7], which leads immediately to th e statement of th e lemma. W e have dφ dt = p · ∇ F ( t p ) = X i p i ∂ F ∂ x i    x = t p . By properties o f th e mu ltilinear extension , we have ∂ F ∂ x i    x = t p = E [ f ( R + i ) − f ( R − i )] ≥ E [ f R ( i )] , where R is a random set sampled indepen dently with probab ilities x i = tp i (see [7] for more details). Therefo re, dφ dt = X i p i ∂ F ∂ x i    x = t p ≥ X i p i E [ f R ( i )] = E [ X i p i f R ( i )] ≥ E [ f ∗ ( p ) − f ( R )] by the deﬁnition of f ∗ ( p ) . Finally , E [ f ( R )] = F ( t p ) = φ ( t ) , hence we obtain the following differential inequality: dφ dt ≥ f ∗ ( p ) − φ ( t ) under the initial condition φ (0) ≥ 0 . W e solve this as follows: d dt ( e t φ ( t )) = e t φ ( t ) + e t dφ dt ≥ e t f ∗ ( p ) wh ich implies that e b φ ( b ) ≥ e 0 φ (0) + Z b 0 e t f ∗ ( p ) dt ≥ ( e b − 1 ) f ∗ ( p ) . Considering that φ ( b ) = F ( b p ) and f ∗ ( p ) ≥ f + ( p ) , this proves the lemm a. A more ﬁne-grain ed an alysis lead s t o the following stren gthened version of Lemma 4 .9, whose p roof can be found in Appendix B. Lemma 4.1 0. If f : 2 N → R + is a mo notone submodu lar function, F : [0 , 1] N → R + its multilinear extension, and f + : [0 , 1] N → R + its concave closur e, then for any b ∈ [0 , 1] and p ∈ [0 , 1] N , F ( b · p ) ≥  1 −  1 − b n  n  f + ( p ) . Lemma 4.10 implies (13), and ther efore completes the proof of Theo rem 4.8, by setting f = r y and b · p = x . Notice th at the multilinear extension of r y ev aluated at x is E [ r y ( R )] . Furth ermore , r y ( p ) = P i ∈ N y i p i = P i ∈ N y i x i b if p is in the matro id polytope. Hence we obtain (13): E [ r y ( R )] ≥  1 −  1 − b n  n  r + y ( p ) =  1 −  1 − b n  n  X i ∈ N y i x i b = 1 −  1 − b n  n b X i ∈ N y i x i . Theorem 4.3 also shows that an efﬁcient algor ithm for computing r y results in an e fﬁciently implementable near- optimal C R scheme. It is well-k nown that a simple gre edy algor ithm ca n be used to com pute r y ( S ) (in fact an indepen dent set S ′ ⊆ S of maxim um weight): Start with S ′ = ∅ , co nsider th e eleme nts of S in no n-increa sing order of their weight and add the current elem ent i to S ′ if S ′ + i is independen t, otherwise discard i . Moreover , it is ea sy to see that this algorithm is monoton e — th e ordering of the elements by weight does not depend on the set S and hence if an elem ent i is includ ed wh en e valuating r y ( A ) then it will be included in ev aluating r y ( B ) for any B ⊂ A . W e thus obtain our main result for C R schemes in the context o f matroids by combining T heorem 4.3 for a choice of ǫ satisfying ǫ ≤ b 10 n with Theorem 4.8, and by using the inequality (1 − b n ) n ≤ e − b − b 2 10 n 11 . 11 This inequ ality can be obtaine d by observing that 1 − x + x 2 3 ≤ e − x for x ∈ [0 , 1] , and hen ce (1 − b n ) n ≤ ( e − b n − b 2 3 n 2 ) n . L et y = e − b n and z = b 2 3 n 2 for simpli city . One can easily check that for these va lues of y and z we have ( y − z ) n ≤ y n − n y n − 1 z + n 2 2 y n − 2 z 2 . E xpanding the last expre ssion and using n ≥ 2 , since the inequali ty is tri vially true for n = 1 , the desired inequali ty follo ws. 24 Corollary 4.11 . F or any matr o id M , a nd x ∈ b · P I , the r e is an ef ﬁciently imple mentable  b, 1 − e − b b  -balan ced and monoton e C R scheme . As sh own by the follo wing theo rem, th e C R schemes th at can be o btained ac cording to Corollary 4. 11 are, up to an additive ε , asympto tically optimal. Theorem 4. 12. F or any b ∈ (0 , 1] , th er e is no ( b, c ) -balanced CR scheme for uniform ma tr oid s of r ank one on n elements with c > 1 − (1 − b n ) n b . Pr oo f. Let M = ( N , I ) be the u niform matr oid of rank 1 over n = | N | ele ments, and co nsider the point x ∈ b · P I giv en by x i = b/n for i ∈ N . Le t R be a r andom set containing each elemen t i ∈ N independ ently with pro bability x i . The expected rank of R is g iv en by E [ r ( R )] = 1 − P r[ R = ∅ ] = 1 −  1 − b n  n . (14) Moreover , any ( b, c ) -balanced C R scheme returning a set I ∈ I satisﬁ es E [ | I | ] = X i ∈ N Pr[ i ∈ I ] ≥ X i ∈ N bc n = b c. (15) Since I is an ind epende nt su bset of R we have E [ r ( R )] ≥ E [ | I | ] , and the claim follows by (14) and (15). A simple ( b, 1 − b ) -balance d C R scheme: Here we describe a sub -optimal ( b, 1 − b ) -balan ced C R scheme for matroid polytop es. Its adv antage is t hat it is deterministic , simpler and co mputation ally less expensive than the optimal schem e that requ ires solving a linear prog ram. Mo reover , Lemma 4.13 that is at the heart of the scheme, is of independ ent interest an d may ﬁnd other app lications. A similar lemma was ind epende ntly sho wn in [1 1] (p rior to ou r work but in a different context). Let M = ( N , I ) b e a matro id. For S ⊆ N r ecall that r ( S ) is th e rank of S in M . The sp an of a set S denoted by span ( S ) is the set of all elements i ∈ N such that r ( S + i ) = r ( S ) . Lemma 4.13. If M = ( N , I ) is a matr oid, x ∈ P ( M ) , b ∈ [0 , 1] and R a random set such tha t Pr[ i ∈ R ] = bx i , then ther e is an element i 0 such that Pr[ i 0 ∈ span ( R )] ≤ b . Pr oo f. Let r ( S ) = max {| I | : I ⊆ S & I ∈ I } den ote the rank fu nction of matro id M = ( N , I ) . Since x ∈ P ( M ) , it satisﬁes the rank constraints x ( S ) ≤ r ( S ) . For S = span ( R ) , we get x ( span ( R )) ≤ r ( span ( R )) = r ( R ) ≤ | R | . Recall that R is a random set where Pr[ i ∈ R ] = bx i . W e take the expectation on both sides: E [ x ( span ( R ))] = X i x i Pr[ i ∈ span ( R )] , an d E [ | R | ] = X i Pr[ i ∈ R ] = b X i x i . Therefo re, X i ∈ N x i Pr[ i ∈ span ( R )] ≤ b X i ∈ N x i . This implies that there must be an element i 0 such that Pr[ i 0 ∈ span ( R )] ≤ b . W e re mark th at the ineq uality P i x i Pr[ i ∈ sp an ( R )] ≤ E [ | R | ] has an interesting inter pretation: If x ∈ P ( M ) , we sam ple R with pr obabilities x i , then let S = span ( R ) an d sample again S ′ ⊆ S with probab ilities x i , then E [ | S ′ | ] ≤ E [ | R | ] . W e do not use this in the following, tho ugh. Theorem 4.14. F or any matr oid M and any b ∈ [0 , 1] , there is a d eterministic ( b, 1 − b ) -ba lanced C R scheme. 25 Pr oo f. Let x ∈ P ( M ) and sample R with p robab ilities bx i . W e deﬁne an o rdering of elements as fo llows. By Lemma 4.13, there is a n element i 0 such that Pr[ i ∈ span ( R )] ≤ b . W e p lace i 0 at the end of the or der . The n, since x restricted to N \ { i 0 } is in the matroid polytop e o f M \ { i 0 } , we can recursively ﬁnd an ordering by the s ame rule. If the elem ents ar e lab eled 1 , 2 , . . . , | N | in this or der, we obtain that Pr[ i ∈ span ( R ∩ [ i ])] ≤ b for e very i . In f act, we are in terested in th e event that i is in the sp an of the precedin g elements, R ∩ [ i − 1] . This is a subset of R ∩ [ i ] , and hence Pr[ i ∈ span ( R ∩ [ i − 1 ])] ≤ Pr[ i ∈ span ( R ∩ [ i ])] ≤ b. The C R scheme is as follows : • Sample R with probab ilities b x i . • For each element i , if i ∈ R \ span ( R ∩ [ i − 1]) , then inclu de it in I . Obviously , r ( I ∩ [ i ]) = r ( I ∩ [ i − 1]) + 1 wh enever i ∈ I , so r ( I ) = | I | a nd I is an indep enden t set. T o boun d the probability of appearance of i , ob serve that th e ap pearance of elements in [ i − 1] is independen t of the ap pearan ce of i itself, an d hen ce the events i ∈ R and i / ∈ span ( R ∩ [ i − 1]) are independen t. As we argued, Pr[ i ∈ span ( R ∩ [ i − 1 ])] ≤ b . W e conclude: Pr[ i ∈ I | i ∈ R ] = Pr[ i / ∈ span ( R ∩ [ i − 1])] ≥ 1 − b. T o implement the schem e we need to make Lem ma 4.13 algo rithmic. W e can acco mplish it by r andom sampling. Fix an eleme nt i . Pick a ran dom set R and che ck if i ∈ span ( R ) ; repea t sufﬁciently many times to obtain an accu rate estimate of P r[ i ∈ span ( R )] . W e no te that altho ugh the scheme itself is deterministic once we ﬁn d an o rdering of th e elements, the constructio n of the ordering is randomize d due to the estimation of Pr[ i ∈ span ( R )] v ia sampling. 4.5 Contention r esolution for k napsacks Here we sketch a contention r esolution scheme fo r k napsack constraints. T his essentially fo llows fr om kno wn te ch- niques; we r emark that Kulik, Sh achnai and T amir [34, 3 5] showed how to round a f ractional solution to the pro blem max { F ( x ) : x ∈ P } for any con stant number of knapsack constra ints and any non-negative submod ular function, while losing a (1 − ε ) factor for an arbitrarily sma ll ε > 0 . Ou r g oal is to sho w that these technique s can be im ple- mented in a black-bo x f ashion and integrated in our framework. Let N = { 1 , 2 , . . . , n } and let a 1 , a 2 , . . . , a n ∈ [0 , 1] be sizes of th e n items. T he inde penden ce system ind uced by a sing le knapsack co nstraint is F = { S : P i ∈ S a i ≤ 1 } and its natu ral relaxa tion has a variable x i for 1 ≤ i ≤ n and is deﬁned as P F = { x ∈ [0 , 1 ] n : P i a i x i ≤ 1 } . W e r efer to this as the knapsack polyto pe. W e prove the following lemma. Lemma 4.15. F or any b ∈ (0 , 1 / 2) there is is a monoton e ( b, 1 − 2 b ) -ba lanced CR scheme for the knapsack polyto pe. If, for some δ ∈ (0 , 1 2 ) , a i ≤ δ for 1 ≤ i ≤ n , then for any b ∈ (0 , 1 2 e ) ther e is a monotone ( b , 1 − (2 eb ) (1 − δ ) /δ ) - balance d C R scheme. Further , for a ny 0 < 2 δ < ε < 1 2 , if a i ≤ δ fo r 1 ≤ i ≤ n th en the r e is a mon otone (1 − ε, 1 − e − Ω( ε 2 /δ ) ) -balanc ed C R scheme. Pr oo f. The C R scheme is th e same fo r all the cases and works as follows: given x ∈ b · P F we sample R with probab ilities x i . T o obtain I from R we sort the items from R in an order of decreasing size an d set I to be the largest preﬁx of this sequen ce that ﬁts in the knap sack. Equiv alently , we consider the items from R in an or der of decreasin g size and ad d the cur rent item to I if it maintains feasibility in the knapsack, else we discard it. I t is easy to see that this scheme is monoto ne. First, we consider the general c ase where there are no re strictions on the item sizes. Let N big = { i ∈ N | a i > 1 / 2 } be the b ig items in N an d let N small = N \ N big be th e small item s. The pro bability of a t least on e big elem ent bein g in R is at most 2 b since Pr[ N big ∩ R 6 = ∅ ] ≤ X i ∈ N big x i ≤ 2 X i ∈ N big a i x i ≤ 2 b. 26 The ﬁrst inequality is via the u nion bou nd, the seco nd ineq uality is using th e fact that a i > 1 / 2 for a ll big items, a nd the third inequality follows f rom x ∈ b · P F . Thus Pr[ N big ∩ R = ∅ ] ≥ 1 − 2 b . Fix some j ∈ N . W e need to lower bound Pr[ j ∈ I | j ∈ R ] where I is th e outp ut of the C R scheme we described. First c onsider the case that j is b ig. Since all big items are considered before any small item , j is accep ted if it is the unique big item in R . Since items are included in R indepe ndently , we hav e Pr[ j ∈ I | j ∈ R ] ≥ Pr[( N big \ { j } ) ∩ R = ∅ | j ∈ R ] = Pr[( N big \ { j } ) ∩ R = ∅ ] ≥ Pr[ N big ∩ R = ∅ ] ≥ 1 − 2 b . Now we con sider the c ase that j is small. Since a j ≤ 1 / 2 , j will be accepted if a ( R \ { j } ) ≤ 1 / 2 . For any S ⊆ N , we ha ve E [ a ( S ∩ R )] = P i ∈ S a i Pr[ i ∈ R ] ≤ P i ∈ S a i x i . In particu lar E [ a ( R )] ≤ P i ∈ N a i x i ≤ b . By Markov’ s inequality , P r[ a ( R \ { j } ) > 1 / 2] ≤ Pr[ a ( R ) > 1 / 2 ] ≤ 2 b . Therefo re each small item is accepted with pr obability at least 1 − 2 b . The s ame analysis holds for a simpler C R sch eme based o n an o rdering of elemen ts in R in which all big items are considered before any small item. Now we consider th e ca se that fo r 1 ≤ i ≤ n , a i ≤ δ for some parameter δ ≤ 1 / 2 . Fix some j ∈ N . It is clear that j ∈ I if j ∈ R and P i ∈ R \{ j } a i ≤ 1 − δ . Conditio ned on j ∈ R , the prob ability of this e vent is at least 1 − Pr[ P i ∈ R a i > 1 − δ ] . W e upper bo und Pr[ P i ∈ R a i > 1 − δ ] via Ch ernoff bound s. Let Y i be the in dicator ran dom variable f or i to b e chosen in R ; Pr[ Y i = 1] = x i . Let Y = P i a i Y i . W e hav e E [ Y ] = P i a i x i ≤ b < 1 − δ by feasibility of x . W e are inter ested in Pr[ Y > 1 − δ ] = Pr[ P i ∈ R a i > 1 − δ ] . W e can assume that E [ Y ] = b by adding dummy elements if necessary; this can only increase Pr[ Y > 1 − δ ] . W e u se the standard Cherno ff bound, P r[ Z > (1 + α ) µ ] ≤  e α / (1 + α ) 1+ α  µ where Z is a sum o f random variables in [0 , 1] and µ = E [ Z ] . T o app ly this bo und to o ur setting, we c onsider Z = Y /δ = P i a i Y i /δ = P i Z i where Z i = a i Y i /δ is a random v ariable in [0 , 1] since a i ≤ δ . Thus, Pr[ Y > 1 − δ ] = Pr[ Z > (1 + α ) E [ Z ]] ≤  e α / (1 + α ) 1+ α  µ where µ = E [ Z ] = b/δ an d (1 + α ) = (1 − δ ) /b . Using δ ≤ 1 2 , we obtain that Pr[ Y > 1 − δ ] ≤  e 1 + α  (1+ α ) µ ≤  eb 1 − δ  (1 − δ ) /δ ≤ (2 e b ) (1 − δ ) /δ . Finally , let’ s co nsider the case where b = 1 − ǫ and a i ≤ δ ≤ ǫ 2 ≤ 1 4 for all i . Here we u se the Chern off-Hoeffding bound Pr[ Z > (1 + α ) µ ] < e − α 2 µ/ 3 for α ∈ (0 , 1) and Z being a s um of ra ndom v ariables bo unded by [0 , 1] . W e estimate the prob ability that condition ed on j ∈ R , all of R ﬁts in the knapsack. Since a j ≤ δ ≤ ǫ 2 , this probab ility is Pr " X i ∈ R a i ≤ 1 | j ∈ R # ≥ Pr   X i ∈ R \{ j } a i ≤ 1 − ǫ/ 2   . W e have E [ P i ∈ R \{ j } a i ] = P i ∈ N \{ j } a i x i ≤ 1 − ǫ . W e can in fact assume th at µ = E [ P i ∈ R \{ j } a i ] = 1 − ǫ , by adding du mmy elements that can on ly increase the pro bability of overﬂowing 1 − ǫ/ 2 . Applying the Cherno ff bo und for rando m v a riables bounded by δ ( after rescaling as above), we ob tain Pr   X i ∈ R \{ j } a i > 1 − ǫ/ 2   ≤ P r   X i ∈ R \{ j } a i > (1 + ǫ/ 2 ) µ   ≤ e − ǫ 2 µ/ (12 δ ) = e − Ω( ǫ 2 /δ ) . The (1 − ε, 1 − e − Ω( ε 2 /δ ) ) -balanced C R sch eme is d irectly applicable o nly if th e item sizes are relati vely small compare d to the kn apsack capacity . Howev er , standard enumeration tricks allo w us to apply this scheme to general instances as well. This can be done for any constant number of knapsack constraints. W e f ormulate this as follows. Corollary 4.16. F or a ny con stant k ≥ 1 a nd ε > 0 , there is a con stant n 0 (that dep ends only o n ε ) such that fo r an y submodu lar maximization instance in volving k kn apsack constraints (and possibly other constraints), ther e is a set T of at most n 0 elements and a r esidual instance on the r ema ining elements such that • Any α -appr o ximate solution to the r esidu al instance together wit h T is an α (1 − k ε ) -ap pr oximate solutio n to the original instance. 27 • In the r esidual instance, each kn apsack c onstraint admits a (1 − ε, 1 − ε ) -b alanced C R scheme. Pr oo f. Given ε > 0 , let δ = O ( ε 2 / log(1 /ε )) and n 0 = 1 / ( δ ε ) . Select T greedily from the optimal solu tion, by picking elemen ts as lon g as th eir marginal con tribution is at least δε O P T ; note that | T | ≤ n 0 . W e deﬁne th e residual instance so that S is feasible in th e residual instance iff S ∪ T is f easible in th e orig inal instance. The objec tiv e f unction in the n ew instan ce is g deﬁned by settin g g ( S ) = f ( S ∪ T ) for each set S ⊆ N \ T ; note that g is a non-negative submodu lar function if f is. In additio n, in the residual instance we remove all elements whose size for some kn apsack constraint is more than δ · r where r is the residual ca pacity . The number of such elements in a knapsack can be at most 1 /δ and hence they can co ntribute at most ε O P T ; we forgo this v alue f or each kn apsack. W e o btain a residual instance where all sizes are at mo st δ with the capacities no rmalized to 1 . By Lemma 4.1 5, each knapsack ad mits a (1 − ε, 1 − e − Ω( ε 2 /δ ) ) = (1 − ε, 1 − ε ) -b alanced CRS. An adv antage of this black b ox approach is that knapsack constraints can be c ombined arbitrarily with other types of constraints. They do not af fect the approx imation ra tio signiﬁcantly . Howev er , the enumeratio n stage af fects the runnin g time by an O ( n n 0 ) factor . 4.6 Sparse packing systems W e no w consider pac king constraints of the typ e A x ≤ b , where x ∈ { 0 , 1 } N is the indicator vector o f a solution. W e can assume witho ut loss of generality that the righ t-hand side is b = 1 . W e say that th e system is k -sparse, if each column of A has at most k nonzero en tries (i.e., e ach element pa rticipates in at most k linear co nstraints). T he approx imation algorithm s i n [4] can be seen to give a co ntention resolution scheme for k - sparse packing systems. C R scheme f or k - sparse packing systems: • W e say that element j participates in co nstraint i , if a ij > 0 . W e call an element j big for this constraint, if a ij > 1 / 2 . Otherwise we call element j small for this constraint. • Sample R with probab ilities x i . • For each co nstraint i : if there is exactly one big eleme nt in R that participates in i , mark all the small elem ents in R for this c onstraint for d eletion; otherwise check whether P j ∈ R a ij > 1 an d if so, m ark all eleme nts participating in i f or deletion. • Deﬁne I to b e R minus the elements marked for deletion. Based on the analysis in [4], we obtain the following. Lemma 4.1 7. F or any b ∈ (0 , 1 2 k ) , the above is a mo noton e ( b, 1 − 2 k b ) -bala nced C R scheme for k -spa rse packing systems. Pr oo f. Let x = b · y with y ∈ [0 , 1] N , A y ≤ 1 . Consider a ﬁxed element j ∗ . It appears in R with p robability x j ∗ . W e analyze the probab ility that it is removed due to some constraint where it participates. First, note that whether big or small, elem ent j ∗ cannot b e removed due to a co nstraint i if th e remain ing elements have size less th an 1 / 2 , i.e. if P j ∈ R \{ j ∗ } a ij < 1 / 2 . This is b ecause in th is case, th ere is n o other b ig elem ent particip ating in i , and element j ∗ is either big in which case it survives, or it is small and then P j ∈ R a ij ≤ 1 , i.e. the constraint is satisﬁed. Thus it rem ains to analyze the event P j ∈ R \{ j ∗ } a ij ≥ 1 / 2 . Note that this is independent of item j ∗ appearin g in R . By the feasibility of 1 b x , E [ P j ∈ R \{ j ∗ } a ij ] = P j 6 = j ∗ x j a ij ≤ b . By Markov’ s ine quality , Pr[ P j ∈ R \{ j ∗ } a ij ≥ 1 / 2] ≤ 2 b . So an elem ent is r emoved with probab ility at most 2 b fo r each con straint where it pa rticipates. By the union bound , i t is removed by probability at m ost 2 k b . Recall the notion of width for a p acking s ystem: W = ⌊ 1 max i,j a ij ⌋ , where a ij are the entries of the packing matrix (recall that we normalize the right-han d side to be b = 1 ). Assuming that W ≥ 2 , on e can u se a simpler C R scheme and improve the parameters. C R scheme f or k - sparse packing systems of width W : 28 • Sample R with probab ilities x i . • For each constraint i fo r which P j ∈ R a ij > 1 , mark all elements participating in i fo r deletion. • Deﬁne I to b e R minus the elements marked for deletion. Lemma 4.18 . F or any b ∈ (0 , 1 2 e ) , the above is a monotone ( b, 1 − k (2 eb ) W − 1 ) -balanc ed C R scheme fo r an y k -sparse system of packing constr aints of width W ≥ 2 . Pr oo f. Again, let x = b y with y ∈ [0 , 1 ] N , A y ≤ 1 . Let us c onsider an element j ′ and a constraint i that j ′ participates in. If w e con dition o n j ′ being p resent in R , we have µ i = E [ P j ∈ R \{ j ′ } a ij | j ′ ∈ R ] = P j 6 = j ′ a ij x ij ≤ b . By the width property , we hav e a ij ′ ≤ 1 /W ≤ 1 / 2 . W e use the Chernoff bound fo r a sum X of in depend ent [0 , 1] random variables with µ = E [ X ] : Pr[ X > (1 + δ ) µ ] ≤ ( e δ / (1 + δ ) 1+ δ ) µ ≤ ( e/ (1 + δ )) (1+ δ ) µ , with 1 + δ = (1 − a ij ′ ) /µ i ≥ 1 / (2 b ) . Since our rando m variables are bounded b y [0 , max a ij ] , we obtain by scaling Pr   X j ∈ R a ij > 1 | j ′ ∈ R   = P r   X j ∈ R \{ j ′ } a ij > 1 − a ij ′   ≤  e 1 + δ  (1+ δ ) µ i / max a ij ≤ (2 e b ) (1 − a ij ′ ) / max a ij ≤ (2 e b ) W − 1 . Therefo re, each element is r emoved with pr obability at most (2 eb ) W − 1 for each constrain t where it par ticipates. W e remark that a k -sp arse packing system can b e viewed as t he inter section of multiple kn apsack constraints on the elements wher e each element p articipates in a t mo st k constraints. On e can u se the com position lemma ( Lemma 1.6) and the C R -schemes fo r a single knapsack constraint gi ven by Lem ma 4.15 to o btain C R -schemes for k - sparse packing systems. The scheme s that we d escribed and analyzed above can be seen as direct implemen tations of th e co mposition approa ch. 4.7 UFP in paths and tre es W e con sider the f ollowing routing/p acking problem. Let T = ( V , E ) b e a capacitated tree with u e denoting the capacity o f ed ge e ∈ E . W e are g iv en k distinct node pairs s 1 t 1 , . . . , s k t k with p air i having a no n-negative demand d i . W e assume that the instance satisﬁes the no- bottleneck condition, that is, d max = max i d i ≤ u min = min e u e . W e say that an in stance is a unit-dem and instance if d i = 1 fo r e ach i ∈ N and u e is a non-negativ e in teger for ea ch e ∈ E . Let N = { 1 , . . . , k } , an d for i ∈ N , we den ote by Q i ⊆ E the edg es on the unique path b etween s i and t i in T . W e say that S ⊆ N is r outab le if, when routing d i units of ﬂow from s i to t i over Q i for each i ∈ S , then the to tal ﬂow o n any edge e is at most u e . More formally , S is routab le if X i ∈ S : e ∈ Q i d i ≤ u e ∀ e ∈ E . W e are inte rested in ﬁn ding a routable set S ⊆ N that maximizes some weig ht fun ction on N . T he case of linear weights w as considered in [14]. Here, a weigh t w i ≥ 0 is given for i ∈ N , and the go al is to ﬁnd a routable s et S ⊆ N that maximiz es P i ∈ S w i . A constant factor ap proxim ation has been pr esented for this pro blem [14], an d moreover it is known that the problem is APX-hard ev en for unit-deman ds and un it-weights [27]. W e ar e interested in m ore general submodula r weights. Let I = { S ⊆ N | S is r outable } . The problem we consider is max S ∈I f ( S ) , where f is a given n on-negative submodular function. W e present a C R scheme for this problem that implies a co nstant f actor approximation throu gh our framew ork. W e start by presentin g a C R sch eme for unit demand s, which we then extend to general demand s. A natural (packing) LP relaxatio n for P I has a v a riable x i ∈ [0 , 1] f or each pair i and a constraint P i : e ∈ Q i d i x i ≤ u e for each edge e ; recall that Q i is the set of edges on the unique s i - t i path in T . C R scheme f or unit-demands: 29 • Root T arb itrarily . Let depth of pair s i t i be the depth of the least commo n ancestor o f s i and t i in T . • Let R ⊆ N be rando m set obtained by includin g each i independ ently with probability bx i . • Let I = ∅ . • Consider pairs in R in increasing order of depth. – Add i to I if I ∪ { i } is routable , o therwise reject i . • Output I . The techniques in [9, 14] giv e the follo wing lemma. Lemma 4.19. F or an y b ∈ (0 , 1 3 e ) the above is a ( b, 1 − 2 eb 1 − eb ) -balanc ed C R scheme. Pr oo f. Let x ∈ b · P I . Consider a ﬁxed pair i ∗ and let v b e the least common ancestor of s i ∗ and t i ∗ in the rooted tree T ; n ote th at v could be on e of s i ∗ or t i ∗ . Le t P be the un ique path in T from v to s i ∗ and P ′ be the path from v to t i ∗ . Without loss of generality assume that v 6 = s i ∗ and hence P is non -empty . W e wish to upp er bound Pr[ i ∗ 6∈ I | i ∗ ∈ R ] , that is, the p robab ility that i ∗ is r ejected conditioned on it bein g includ ed in the ran dom set R . The reason that i ∗ gets rejected is that at least one ed ge e ∈ P ∪ P ′ is already f ull f rom the pairs that have been accepted into I prio r to consider ing i ∗ . W e u pper b ound th e probability of this event hap pening f or some edg e in P and use a symmetric argument for P ′ . Let e 1 , e 2 , . . . , e h be the edges in P from v to s i ∗ . Let E j be the e vent that i ∗ gets r ejected at e j , th at is, the capacity of e j is full when i ∗ is considered for addition to I . Note that these events are cor related. W e claim the following: if j > h and u e j ≥ u e h then E j happen s only if E h happen s. The reason fo r this is the order in which th e pairs in R are considered for insertion. When i ∗ is considered, the only pairs inserte d in I pr ior to it are tho se whose depth is n o lar ger, and h ence the total capacity u sed o n an edge decreases as we traverse the path P fr om v to s i . Thus, to analyze the pro bability o f rejection it su fﬁces to consider a subsequen ce of e 1 , e 2 , . . . , e h starting with e 1 such that th e capacity of th e next edge in the s equenc e is strictly sm aller th an the p reviously added one. For notational simplicity we will therefore assume that u e 1 > u e 2 > . . . > u e h ≥ 1 . Let S j = { i 6 = i ∗ | e ∈ Q i } be the set of p airs other tha n i ∗ that contain e in their path Q i . Let E ′ j be the event tha t | R ∩ S j | ≥ u e j . It is easy to see th at P r[ E j ] ≤ Pr[ E ′ j ] . Since 1 b x is a feasible solution to the L P relax ation we have P i ∈ S j x i < bu e j . Letting X i be th e event that i ∈ R , and X = P i ∈ S j X i , we have Pr[ E ′ j ] = Pr[ X ≥ u e j ] . Since X is the sum of independen t [0 , 1] random v ariables X i , and has expectation bu e j , we ob tain by standard Chernoff bound s: Pr[ E ′ j ] = Pr[ X ≥ u e j ] ≤ ( e δ / (1 + δ ) 1+ δ ) µ ≤ ( e/ (1 + δ )) (1+ δ ) µ , where µ = bu e j and δ = 1 /b − 1 . Hen ce, Pr[ E ′ j ] ≤ ( eb ) u e j . T aking the union bo und over all edges in the p ath, the probab ility of rejection of i ∗ on some edg e in P is at mo st P h j =1 ( eb ) u e j ≤ P ∞ ℓ =1 ( eb ) ℓ = eb 1 − eb , whe re the inequ ality is due to the fact th at the edge capacities are strictly decreasing an d lo wer bounded by 1 , and the equality is due to the fact th at eb < 1 (recall that b ∈ (0 , 1 3 e ) ). By a union bound over P an d P ′ we hav e that the pro bability o f i ∗ being rejected cond itioned o n it being in R is at most 2 eb 1 − eb . C R scheme for genera l demands: A C R scheme fo r general dem ands can b e obta ined as follows. Th e linear program P I is a pack ing LP of th e form A x ≤ b , x ∈ [0 , 1 ] where A is colum n-restricted (all the non- zero v alues in a colu mn have the same value). For such colum n-restricted packing integer p rogram s (CPIPs), when demands satisfy the no- bottleneck assumption , one can use grou ping and scaling tech niques ﬁrst sugge sted by Kolliopoulos and Stein [31] (see also [14]) to show that th e integrality g ap for a CPIP with matr ix A is at most a ﬁx ed con stant factor worse than that of the un derlyin g 0 - 1 matrix A ′ (obtained fro m A by placing a 1 in each non -zero entry). Note th at in the context of the U FP problem, the matrix A corresp onds to the pr oblem wit h arbitrary deman ds while the m atrix A ′ correspo nds to the o ne with unit-demands. On e can use the sam e g roupin g and scaling t echniqu es to show th at a monotone ( b, 1 − b ′ ) - balanced C R scheme for A ′ can be used to obtain a mono tone ( b/ 6 , (1 − b ′ ) / 2) -balanc ed C R scheme for A . W e give a pr oof in Section 4 .8, see Theorem 4. 20. Using this general co n version theorem and Lemma 4.19, o ne c an o btain a ( b, b ′ ) -balanced C R scheme for UFP in trees fo r some sufﬁciently small but absolute constan ts b and b ′ . This sufﬁces 30 to obtain a con stant factor approx imation for maximizing a no n-negative subm odular fu nction o f routable r equests in a cap acitated tree. Howe ver , the ( b/ 6 , (1 − b ′ ) / 2) -balanc ed C R scheme does not a llow comp osition with other constraints via Lemma 1.6 sinc e (1 − b ′ ) / 2 do es no t ten d to zero even if b ′ does. Howe ver , T heorem 4.20 gives a more reﬁned statement that is helpfu l in applications in light of Remark 1.8. W ithout the no -bottlenec k assumption, the lin ear pro gram has an Ω( n ) integrality ga p even for UFP o n paths [ 9]. One can still apply th e g roup ing and scaling techniques without the no-bo ttleneck assumption under a mild restriction; we refer the reader to [13]. 4.8 Column-r estricted packing constraints Here we co nsider C R scheme s for CPIPs. W e follow the notation fro m [14]. Le t A be an arbitr ary m × n { 0 , 1 } - matrix, an d d be an n -element non-n egati ve vector with d j denoting the j th entry in d . Let A [ d ] denote the matrix obtained by multiply ing e very entry of column j in A by d j . A CPIP is a problem of the form max w x , subject to A [ d ] x ≤ b , x ∈ { 0 , 1 } n . Note tha t all non-zero entries in A [ d ] for any gi ven column have the sam e value an d hen ce the n ame colu mn-restricte d. Here we are in terested in submodu lar objecti ve functions an d the g oal is obtain a C R scheme for the polytope P I induced by the relaxation A [ d ] x ≤ b , x ∈ [0 , 1 ] n . Instead of focusing on th e polytope for a given d and b , we consider the class of polytop es induced b y all d, b . Theorem 4.20 . Supp ose ther e is a mono tone ( β , 1 − β ′ ) C R scheme for th e p olytope A x ≤ b , x ∈ [0 , 1] n for every b ∈ Z + wher e A is { 0 , 1 } -matrix. Then there is a mo noton e ( β / 6 , (1 − β ′ ) / 2) -bala nced C R scheme for th e polytope A [ d ] x ≤ b , x ∈ [0 , 1 ] n for all d, b such that d max = max j d j ≤ b min = min j b j . Mor eover ther e is a monoto ne ( β / 6 , 1 − β ′ ) -balanc ed C R scheme if all d j ≤ b min / 3 or if all d j ≥ b min / 3 . W e sketch the proo f of the above theorem which fo llows the grouping a nd scaling id eas previously used in [ 31, 14]. W e have chosen some speciﬁc constants in th e th eorem f or simp licity . One can ob tain som e gen eralizations and variations o f the above theorem via the same ideas. Let N = { 1 , . . . , n } b e a gro und set corr espondin g to th e colum ns. Given d , for integer h ≥ 0 we let N h = { j ∈ N | d j ∈ ( d max / 3 h +1 , d max / 3 h ] } . W e think of the colu mns in N 0 as larg e an d th e rest as small . The overall id ea is to fo cus e ither on the large deman ds or the small deman ds. Moreover, we will see that small d emands c an b e tre ated indepen dently within each g roup N h . Let z be a feasible solu tion to the system A [ d ] x ≤ b , x ∈ [0 , 1] n . For integer h ≥ 0 we let z h denote the vector obtain ed from z as follo ws: z h j = z j / 6 if j ∈ N h and z h j = 0 otherwise. Th e vector z h restricts the solution z to elements in N h and scales it down by a small constant factor . W e also deﬁne a correspo nding vector b h where b h i = ⌈ A i z h ⌉ for each r ow i . W e have the following lemma which is a restateme nt of correspo nding s tatements from [31, 14]. Lemma 4.21. F or h ≥ 0 , let y h ∈ { 0 , 1 } n be a feasible inte gral solution to A x ≤ b h , x ∈ [0 , 1] n such that y j = 0 if z h j = 0 . Then A [ d ] y 0 ≤ b and P h ≥ 1 A [ d ] y h ≤ b . Pr oo f. Fix some h and consider the i -th row of A [ d ] y h which is equal to P j ∈ N h d j A ij y h i . W e upper bound this quantity as follows: X j ∈ N h d j A ij y h i ≤ d max 3 h X j ∈ N h A ij y h i (from deﬁnition of N h ) ≤ d max 3 h b h i (feasibility of y h ) ≤ d max 3 h   1 + X j ∈ N h A ij z h i   (deﬁnition of b h and using ⌈ a ⌉ ≤ 1 + a ) ≤ d max 3 h   1 + X j ∈ N h A ij z j / 6   (from deﬁnition of z h ) ≤ d max 3 h + 1 2 X j ∈ N h A ij d j z j ( d j > d max / 3 h +1 for j ∈ N h ) . 31 For h = 0 we need a slight variant of the above where we replace b h i by ma x { 1 , 2 P j ∈ N 0 A ij z h i } since ⌈ a ⌉ ≤ max { 1 , 2 a } . Th en we obtain that X j ∈ N 0 d j A ij y 0 i ≤ ma x { d max , X j ∈ N o A ij d j z j } ≤ b i , since d max ≤ b min and z is feasible. Thus A [ d ] y 0 ≤ b . For t he second part of the claim, conside r a ro w i . X h ≥ 1 X j ∈ N h d j A ij y h i ≤ X h ≥ 1   d max 3 h + 1 2 X j ∈ N h A ij d j z j   ≤ X h ≥ 1 d max 3 h + X h ≥ 1 1 2 X j ∈ N h A ij d j z j ≤ d max 2 + b i 2 ≤ b i . The penultimate ineq uality is from the feasibility of z , and th e last inequ ality is from the assumptio n that d max ≤ b min . W ith th e above claim in plac e we can describe the C R sch eme claimed in the theorem . Le t z be a feasible so lution and let z h for h ≥ 0 be construc ted from z as describe d above. C R scheme: • For each h ≥ 0 in depend ently run the ( β , 1 − β ′ ) -balanced CR scheme for the polyto pe A x ≤ b h , x ∈ [0 , 1 ] n with fractional solution z h to obtain integral vectors y h , h ≥ 0 . • W ith probability 1 / 2 outp ut y 0 , otherwise output P h ≥ 1 y h . W e claim that th e above scheme is a monotone ( β / 6 , (1 − β ′ ) / 2) -balanc ed C R sche me. Note that we use the unit-dem and scheme in a black -box fashion. First, we observe via Lemma 4.21 that the o utput of th e schem e is a feasible inte gral solutio n. An alternativ e description of th e schem e is as f ollows. W e ar e g iv en a point x = β 6 z with z ∈ [0 , 1] n , A z ≤ b . Obtain a set R ⊆ N by indepe ndently samplin g e ach j ∈ N with probability x j = β / 6 · z j . Let R h = R ∩ N h . For each h o btain I h ⊆ R h as the output o f the scheme for A y ≤ b h , y ∈ [0 , 1] n giv en the random set R h . W ith p robab ility 1 / 2 outpu t I = I 0 otherwise o utput I = ∪ h ≥ 1 I h . For j ∈ N h we h ave tha t Pr[ j ∈ I h | j ∈ R h ] ≥ 1 − β ′ . Further, P r[ j ∈ I | j ∈ I h ] = 1 / 2 by the choice o f the algo rithm in the second step. Therefo re Pr[ j ∈ I | j ∈ R ] ≥ (1 − β ′ ) / 2 . It is easy to verify the scheme is monotone. Further, if we only have large demands or only small demand s the n the second step is not necessary and h ence we obtain a ( β / 6 , (1 − β ′ )) -balanced C R scheme. Acknowledgments: W e thank Mohit Singh for help ful discussions on contentio n resolution schemes for matroids, and Shipra Agrawal for discussions concer ning the correlation gap. W e thank two anonym ous reviewers for th eir comments which helped us improve the pr esentation of the details in the paper . Refer ences [1] A. Ageev and M . Sviridenko. Pip age rounding : a new method of constructing algorithms with proven per for- mance guarantee. J. of C ombina torial Optimization , 8:307–328 , 20 04. [2] S. Agrawal, Y . Ding, A. Saberi an d Y . Y e. Price of Correlations in Stochastic Optimization. Operations Researc h , 60(1) :150–1 62, 20 12. Prelim inary version with a d ifferent title in Pr oc. of 2 1 st AC M-SIAM S OD A , 1087 –109 6, 2010. 32 [3] N. Alon and J. Spencer . The Probabilistic Method. 3rd Edition, John W iley & Sons, 2008 . [4] N. Bansal, N. K orula, V . Nagarajan and A. Sriniv asan. Solving Packing Integer Pro grams via Randomized Rounding with Altera tions. Theory of Computing , 8 (1): 53 3–56 5, 2012. Preliminar y version with a different title in Pr oc. of 1 4 th IPCO , 369 –382 , 201 0. [5] G. Caline scu, A. Chakrabar ti, H. Karloff an d Y . Raban i. Improved approximation algorithms for resource al- location. ACM T ransaction s on Algorithms , 7(4), 2 011. Preliminary version in Pr oc. of 8 th IPCO , 439– 456, 2001. [6] G. Calinescu, C. Chekuri, M. P ´ al and J. V o ndr ´ ak. Max imizing a submodula r set f unction subject to a matroid constraint. Pr oc. of 12 th IPCO , 182–1 96, 2007. [7] G. Calinescu, C. Ch ekuri, M. P ´ al an d J. V ondr ´ ak . Max imizing a mon otone submod ular set functio n subject to a matroid constraint. SIAM Journal on C omputing 40:6 (specia l section on ST OC 2008) , 1 740– 1766 , 2011. [8] A. K. Chandr a, D. S. Hirschberg, an d C. K. W o ng. Approximate a lgorithms for some generalize d k napsack problem s. Theo r etical Computer Science , 3(3):293 –304 , 1976. [9] A. Cha krabarti, C. Chekur i, A. Gu pta and A. Kumar . Ap proxim ation algorithms fo r th e un splittable ﬂow pro b- lem. Algorithmica , 47(1):53– 78, 200 7. [10] T . C han and S. Har-Peled. Approx imation algorith ms fo r m aximum ind epende nt set of p seudo-d isks. Discrete & Computationa l Geo metry , 48(2) :373–3 92, 2012. Preliminary version in Pr oc. of ACM SoCG , 2 009. [11] S. Chawla, J. Ha rtline, D. Malec and B. S iv an. Seque ntial posted p ricing and mu lti-parame ter mechanism design. Manuscript, a vailable at http://arxiv.org/ab s/0907.2435v 1 , 2009. [12] S. Chawla, J. Hartline, D . Malec and B. Siv an. Multi-par ameter Mecha nism Design an d Sequential Posted Pricing. Pr o c. of 42 nd AC M STOC , 311–320 , 201 0. [13] C. Chekuri, A. E ne and N. K orula. UFP in paths and tree s and column-restricted packing integer pro grams. P r oc . of 12 th APPRO X , 42–55, 2009. [14] C. Chekuri, M. Mydlarz and F . B. Shepherd. Multico mmodity d emand ﬂo w in a tree and packing integer p ro- grams. ACM T r ans. on Algorithms , 3(3) , 2007. P reliminary version in Pr oc. of ICALP , 41 0–42 5, 20 03. [15] C. Chekuri, J. V ond r ´ ak an d R. Zen klusen. Depen dent ra ndomize d round ing v ia e xchang e properties of com bina- torial structures. Pr o c. of 51 st IEEE FOCS , 575– 584, 2010. [16] C. Cheku ri, J. V ond r ´ ak and R. Zenklusen. Submodu lar function maximization via the multilinear relaxation and contention resolution schemes. Pr oc. of 43 r d AC M STOC , 783–792 , 201 1. [17] C. Cheku ri, J. V ond r ´ ak and R. Zenklusen. Submodu lar function maximization via the multilinear relaxation and contention resolution schemes. Manuscr ipt, a vailable at http ://arxiv.or g/abs/1105. 4593v2 , 2011 . [18] G. Corn uejols, M. L. Fisher and G . Nem hauser . Lo cation of ba nk acco unts to o ptimize ﬂoat: An analy tic study of exact and approximate algorithms. Management S cience , 23:789– 810, 19 77. [19] A. Ene, S. Har -Peled and B. Raich el. Geo metric pack ing u nder n on-un iform constraints. Pr oc. of ACM SoCG , 2012. [20] U. Feige. A thre shold of ln n f or approx imating s et cover . J ournal of the A CM , 45(4) :634–6 52, 1998 . [21] U. Feige, V . Mirrok ni and J. V ondr ´ ak . Maximizing non-mon otone sub modu lar f unction s. SIAM J ournal on Computing , 40(4): 1133– 1153 , 2011. Preliminary version in Pr oc. of 48 th IEEE FOCS , 461– 471, 2007. [22] U. Feige and J. V ond r ´ ak. Approxim ation algo rithms fo r allo cation p roblems: Improving th e factor of 1 − 1 /e . Pr oc . of 47 th IEEE FOCS , 667– 676, 2006. 33 [23] U. Feige an d J. V ondr ´ ak. The submod ular welfare p roblem with deman d queries. Theory of Computing , 2 47–29 0, 2010. [24] M. Feldm an, J. Nao r , R. Schwartz. A un iﬁed continuo us greedy algorithm for submo dular maximization . Pr oc. of 52 nd IEEE FOCS , 570–57 9, 20 11. [25] M. L. Fisher, G. L. Nemhauser and L. A. W olse y . An analysis of appr oximation s for maximizing submo dular set function s - I I. Math. Pr og. Stud y , 8:73–87, 1978. [26] A. M. Friez e an d M. R. B. Clarke. App roxima tion algorithm s for th e m -dimensional 0 - 1 knapsack p roblem: worst-case and probabilistic analyses. Eur op ean Journal o f Operational Resear ch , 15(1):10 0–109 , 1984. [27] N. Garg. V . V . V a zirani, M. Y annakakis. Primal-d ual approximatio n algorithm s for inte gral ﬂow and multicut in trees. Algorithmica 18(1 ):3–20 , 199 7. Preliminary version in P r oc. of ICALP , 199 3. [28] A. Gupta, V . Nagarajan and R. Ra v i. Robust and MaxMin optimization under matroid and knapsack uncertainty sets. Manuscrip t, http://arxiv.org/a bs/1012.496 2v2 , 2010. [29] A. Gup ta, A. Roth, G. Scho enebeck and K. T alw ar . Constrained non-mo noton e sub modular m aximiza- tion: Ofﬂine and secretary algor ithms. Pr oc. of WINE , 20 10. A mo re d etailed version av ailable at http://arxiv .org/abs/10 03.1517v2 . [30] J. H ˚ astad. Clique is hard to app roxima te within n 1 − ε . Acta Mathematica , 182:1 05–1 42, 199 9. [31] S. G. K o lliopoulo s and C. Stein. App roximatin g disjoint-path problems using gr eedy algor ithms and Packing Integer Programs. Math. Pr ogramming A , (99):63–8 7, 2 004. Preliminary version in Pr oc. of IPCO , 1998 . [32] B. Korte and J. Vy gen. Combina torial Optimization: Theory and Algorithms . Springer, 2012 . [33] A. Kulik, H. Shachn ai an d T . T amir . Maximizing s ubmod ular set fun ctions subjec t to multip le linear c onstraints. Pr oc . of 20 th AC M-SIAM SOD A , 545–55 4, 20 09. [34] A. Kulik, H. Sh achnai and T . T amir . Ap prox imations for m onoton e and n on-mo noton e subm odular maximization with knapsack constraints. Manuscrip t, 2010. [35] A. Kulik, H. Shachnai an d T . T amir . Approxima tions for monotone and nonmon otone submodular ma ximization with knapsack constraints. Math. of Operations Resear ch , 38( 4):729 –739 , 2013. [36] J. Lee, V . Mirrokn i, V . Nagarajan and M. Sviridenko. Maximizing nonm onoto ne su bmodu lar functions under matroid and knapsack constraints. SI AM J . on Disc. Math. , 2 3(4): 205 3–207 8, 20 10. Preliminary version in Pr oc . of 41 st AC M STOC , 323–3 32, 2 009. [37] J. Lee, M. Sviriden ko and J. V ond r ´ ak. Sub modu lar maximization over multiple matroids v ia generalized ex- change p roperties. Math. of Operations Researc h , 35, 7 95–8 06, 201 0. Preliminar y version in Pr oc. o f 1 2 th APPRO X , 244–257, 2009. [38] J. Lee, M. Sviriden ko and J. V o ndr ´ ak . Matr oid matching : the power o f local search. SIAM J. on Computing , 42(1) :357–3 79, 2013. Preliminary version in Pr oc. of 42 th AC M STOC , 369–3 78, 2 010. [39] L. Lov ´ asz. Submodu lar function s and co n vexity . Math. Pr og.–the state of the art (Bonn 1982) , 235–25 7, 19 83. [40] V . Mirrokni, M. Schapira and J. V ondr ´ ak. T ig ht information- theoretic lower b ounds for welf are maximization in combinato rial auctions. P r oc. of 9 th AC M EC , 70–77 , 2 008. [41] G. L. Nemh auser and L. A. W olsey . Best algorithms for appr oximating the max imum of a submo dular set function . Ma th. Oper . Resear ch , 3(3):177–1 88, 1978. [42] G. L. Nemh auser, L. A. W olsey and M. L. Fisher . An analysis of appro ximation s for m aximizing submodular set function s - I . Math. Pr og. , 1 4:265 –294 , 1978. 34 [43] S. Oveis Gh aran and J. V ond r ´ ak. Sub modu lar maximization by simu lated ann ealing. Pr oc. of 2 2 nd AC M-SIAM SOD A , 1098– 1116, 2 011. [44] A. Schrijver . Combina torial Optimization - P o lyhedra and Ef ﬁciency . Spr inger, 2003 . [45] A. Srini v asan. New ap proach es to covering an d p acking pro blems. Pr oc. o f 12 th AC M-SIAM S OD A , 56 7–57 6, 2001. [46] M. Sviridenko. A n ote on maximizing a submodu lar set fun ction subject to kna psack constraint. Operations Resear ch Letters 32 , 41–43, 2004. [47] L.G. V aliant. The complexity of enumeration and reliability problems. S IAM Journal on Compu ting 8( 3):410 - 421, 1979. [48] J. V ondr ´ ak . Sub modu larity in c ombinato rial optimization. Ph D Thesis , Charles Univ ersity , 2007. [49] J. V ond r ´ ak. Optimal approximation f or the subm odular welf are problem in the v alue or acle model. Pr oc. of 40 th AC M STOC , 67–74 , 2008. [50] J. V ondr ´ ak. Symm etry and approx imability of sub modular maximization problems. SIA M J . on Computing , 42(1) : 26 5–30 4, 2013. Pre liminary version in Pr oc. of 50 th IEEE FOCS , 251–27 0, 20 09. [51] J. V on dr ´ ak. Symmetry and ap proxim ability of submodu lar max imization p roblems. Manuscript, available at http://arxiv .org/abs/11 10.4860v2 , 201 3. [52] J. V o ndr ´ ak. A note on concentr ation of submo dular fun ctions. Manuscript, av ailable at http://arxiv .org/abs/10 05.2791v1 , 201 0. [53] Q. Y a n. Mechan ism design via correlation gap. P r oc. of 22 th AC M-SIAM SOD A , 710–71 9, 20 11. A A ppr o ximation f or general polytopes In this section, we fo rmulate an approx imation result for the prob lem max { F ( x ) : x ∈ P } when P is a ge neral solvable po lytope (not n ecessarily down-monotone) . T his result is included o nly for th e sake of co mpleness; we do not have any concrete applications for it. Our result generalizes (while losing a factor of 4) the result for matroid base polytop es fr om [50], which states that a 1 2 (1 − 1 ν − o (1)) -app roximatio n can be achieved, p rovided that the fractional base p acking num ber is at least ν where ν ∈ [1 , 2] . As observed in [50], the fractional b ase p acking num ber being at least ν is eq uiv alent to the condition P ∩ [0 , 1 ν ] N 6 = ∅ . This is the conditio n we use for g eneral polytopes. W e state the algorithm only in its continu ous form; we o mit the discretization details. Algorithm A.1. Let t ∈ [0 , 1] be a parameter such that P ∩ [0 , t ] N 6 = ∅ . Initialize x ∈ P ∩ [0 , t ] N arbitrarily . As long as ther e is y ∈ P ∩ [0 , 1 2 (1 + t )] N such that ( y − x ) · ∇ F ( x ) > 0 (which can be found by l inear pr ogramming ), move x con tinuously in the dir e ction y − x . If there is n o such y ∈ P ∩ [0 , 1 2 (1 + t )] N , r eturn x . Note that even thou gh we requ ire P ∩ [0 , t ] N 6 = ∅ , the lo cal search works inside a larger poly tope P ∩ [0 , 1 2 (1 + t )] N . This is necessary for the analysis. Theorem A.2 . F or a ny solvab le p olytope su ch tha t P ∩ [0 , t ] N 6 = ∅ , Alg orithm A .1 ap pr o ximates the p r o blem max { F ( x ) : x ∈ P } within a factor of 1 8 (1 − t ) . Pr oo f. The alg orithm maintains the in v ariant x ∈ P ∩ [0 , 1 2 (1 + t )] N . Suppo se that the a lgorithm returns a point x . Then we know that for e very y ∈ P ∩ [0 , 1 2 (1 + t )] N , ( y − x ) · ∇ F ( x ) ≤ 0 . W e use a particular point y deﬁned as follows: Let x ∗ be the optimum, i.e. F ( x ∗ ) = max { F ( x ) : x ∈ P } , and let x 0 be any po int in P ∩ [0 , t ] N , for example the starting point. Th en we d eﬁne y = 1 2 ( x 0 + x ∗ ) . By conv exity , we ha ve y ∈ P , and since x ∗ ∈ [0 , 1] N , we also ha ve y ∈ [0 , 1 2 (1 + t )] N . Therefore, b y the local-search condition, we ha ve ( y − x ) · ∇ F ( x ) ≤ 0 . By Le mma 3. 2, 2 F ( x ) ≥ F ( x ∨ y ) + F ( x ∧ y ) ≥ F ( x ∨ y ) . 35 Let x ′ = x ∨ y . The point x ′ has the fo llowing prope rties: x ′ = x ∨ 1 2 ( x 0 + x ∗ ) ≥ 1 2 x ∗ , and also x ′ ∈ [0 , 1 2 (1 + t )] N . Consider the ray 1 2 x ∗ + ξ ( x ′ − 1 2 x ∗ ) par ameterized by ξ ≥ 0 . Ob serve that this ray has a po siti ve direction in all coordin ates, and it is possible to go beyond ξ = 1 and still s tay i nside [0 , 1] N : in particular , for ξ = 2 1+ t we get a point 1 2 x ∗ + 2 1+ t ( x ′ − 1 2 x ∗ ) ≤ 2 1+ t x ′ ∈ [0 , 1] N . Using this fact, we can express x ′ as a conve x comb ination: x ′ = 1 + t 2 ·  1 2 x ∗ + 2 1 + t ( x ′ − 1 2 x ∗ )  + 1 − t 2 · 1 2 x ∗ (the reader can verify that this is an identity). By the concavity o f F in positive direc tions, we get F ( x ′ ) ≥ 1 + t 2 F  1 2 x ∗ + 2 1 + t ( x ′ − 1 2 x ∗ )  + 1 − t 2 F  1 2 x ∗  . As we argued, 1 2 x ∗ + 2 1+ t ( x ′ − 1 2 x ∗ ) ∈ [0 , 1] N , so we can just lower -boun d the respective value by 0 , and we obtain F ( x ′ ) ≥ 1 − t 2 F  1 2 x ∗  ≥ 1 − t 4 F ( x ∗ ) . Finally , our solution satisﬁes F ( x ) ≥ 1 2 F ( x ∨ y ) = 1 2 F ( x ′ ) ≥ 1 − t 8 F ( x ∗ ) = 1 − t 8 O P T . B Missing proofs of Section 4 Pr oof of Theor em 4.1 Pr oo f. As ob served in th e proo f o f Theor em (1.3), it sufﬁces to show (1 0) (assuming an arbitra ry ordering of the elements N = { 1 , . . . , n } ). Let us take the e xpectation in two steps, ﬁrst o ver I condition ed on R , and then over R : E [ f ( I ∩ [ i ]) − f ( I ∩ [ i − 1])] ≥ E R [ E I [ 1 i ∈ I f R ∩ [ i − 1] ( i ) | R ]] = E R [Pr[ i ∈ I | R ] f R ∩ [ i − 1] ( i )] . Note that Pr[ i ∈ I | R ] can be nonzer o only if i ∈ R , ther efore we can restrict our attention to this e vent: E [ f ( I ∩ [ i ]) − f ( I ∩ [ i − 1])] ≥ Pr[ i ∈ R ] · E [Pr [ i ∈ I | R ] f R ∩ [ i − 1] ( i ) | i ∈ R ] . On th e pr oduct space assoc iated with the distrib ution of R conditioned on i ∈ R , b oth Pr[ i ∈ I | R ] and f R ∩ [ i − 1] ( i ) are n on-in creasing functions, due to I being monotone with r espect to R , and f being sub modu lar . Ther efore, th e FKG inequality (see [3]) implies that E R [Pr[ i ∈ I | R ] f R ∩ [ i − 1] ( i ) | i ∈ R ] ≥ E R [Pr[ i ∈ I | R ] | i ∈ R ] · E R [ f R ∩ [ i − 1] ( i ) | i ∈ R ] = P r[ i ∈ I | i ∈ R ] · E [ f R ∩ [ i − 1] ( i )] . since the m arginal v alue f R ∩ [ i − 1] ( i ) does n ot depend on i ∈ R . By the ( b, c ) -balanced pro perty , Pr[ i ∈ I | i ∈ R ] ≥ c ; in additio n, f is eith er mon otone o r we assume that Pr[ i ∈ I | i ∈ R ] = c . In both cases, Pr[ i ∈ I | i ∈ R ] · E [ f R ∩ [ i − 1] ( i )] ≥ c · E [ f R ∩ [ i − 1] ( i )] . W e summarize: E [ f ( I ∩ [ i ]) − f ( I ∩ [ i − 1])] ≥ Pr[ i ∈ R ] · c E [ f R ∩ [ i − 1] ( i )] = c E [ f ( R ∩ [ i ]) − f ( R ∩ [ i − 1])] . Therefo re, E [ f ( I )] = f ( ∅ ) + n X i =1 E [ f ( I ∩ [ i ]) − f ( I ∩ [ i − 1])] ≥ f ( ∅ ) + c n X i =1 E [ f ( R ∩ [ i ]) − f ( R ∩ [ i − 1])] ≥ c E [ f ( R )] . 36 Pr oof of Lemma 4.10 T o prove L emma 4 .10 we u se a p roperty of sub modula r function s p resented in [48], wh ich is stated b elow as Lemm a B.1. Lemma B.1 ([48]) . Let f : 2 N → R + be a monotone submodular function, and A 1 , . . . , A m ⊆ N . F or each j ∈ [ m ] indepen dently , sample a rando m subset A j ( q j ) whic h con tains each element o f A j with pr obability q j . Let J be a random subset of [ m ] conta ining each elemen t j ∈ [ m ] independen tly with pr obability q j . Then E [ f ( A 1 ( q 1 ) ∪ · · · ∪ A m ( q m ))] ≥ E   f   [ j ∈ J A j     . Lemma B.2 below is a g eneralization of L emma 4. 2 in [4 8]. W e follow the same pr oof techniq ue as used in [48]. The lemma contains two statements. The ﬁrst is a simpler statemen t that may be of indepen dent interest. The second, which can be seen to b e a slightly stro nger version of the ﬁrst stateme nt, turn s out to imply Lem ma (4.10), as we will show i n the following. Lemma B.2. Let f : 2 N → R + be a monoto ne submo dular function , and A 1 , . . . , A m ⊆ N . F or each j ∈ [ m ] indepen dently sample a random subset A j ( q j ) which contain s ea ch eleme nt of A j with pr obability q j . Let q = P m j =1 q j . Then E [ f ( A 1 ( q 1 ) ∪ · · · ∪ A m ( q m ))] ≥ 1 q  1 −  1 − q m  m  m X j =1 q j f ( A j ) . Furthermore for m ≥ 2 a nd any s, t ∈ [ m ] with s 6 = t , E [ f ( A 1 ( q 1 ) ∪ · · · ∪ A m ( q m ))] ≥ 1 q − q s q t 1 −  1 − q − q s q t m − 1  m − 1 !   − q s q t min { f ( A s ) , f ( A t ) } + m X j =1 q j f ( A j )   . Pr oo f. Observe that the ﬁr st statement is a consequence o f the second one: it sufﬁces to add an arbitrary ad ditional set A m +1 with probability q m +1 = 0 to the family of sets and in voke the second p art of the lemma with s = 1 and t = m + 1 . Hence, we only prove the second part of the lemma. By L emma B.1 it sufﬁces to estimate E [ f ( ∪ j ∈ J A j )] , where J is a rando m subset o f [ m ] co ntaining elemen t j ∈ [ m ] indepen dently of the oth ers with pro bability q j . Assume f ( A 1 ) ≥ · · · ≥ f ( A m ) and with out loss o f generality we assume t > s . W e d eﬁne for k ∈ [ m ] , J k = { I ⊆ [ m ] | min( I ) = k } . By mono tonicity o f f , we have f ( ∪ j ∈ J A j ) ≥ f ( A k ) if J ∈ J k . Hence, E [ f ( ∪ j ∈ J A j )] ≤ m X j =1 Pr[ J ∈ J j ] f ( A j ) = m X j =1 f ( A j ) q j j − 1 Y ℓ =1 (1 − q ℓ ) . Thus, it sufﬁces to p rove m X j =1 f ( A j ) q j j − 1 Y ℓ =1 (1 − q ℓ ) ≥ 1 q − q s q t 1 −  1 − q − q s q t m − 1  m − 1 !   − q s q t f ( A t ) + m X j =1 q j f ( A j )   . (16) Since the above inequality is linear in the parameter s f ( A j ) , it sufﬁces to prove it for th e special case f ( A 1 ) = · · · = f ( A r ) = 1 and f ( A r +1 ) = · · · = f ( A m ) = 0 (A gen eral decreasing sequence of f ( A j ) can be obtained as a positi ve linear combin ation of such special cases.) Hen ce, it remains to prove r X j =1 q j j − 1 Y ℓ =1 (1 − q ℓ ) ≥ 1 q − q s q t 1 −  1 − q − q s q t m − 1  m − 1 !   − 1 r ≥ t · p s p t + r X j =1 q j   , (17) 37 where 1 r ≥ t is equal to 1 if r ≥ t and 0 o therwise. T o pr ove (17) we distinguish two cases de pending on wheth er r < t or r ≥ t . Case r < t : Expan ding the left-hand side of (17), we obtain r X j =1 q j j − 1 Y ℓ =1 (1 − q ℓ ) = 1 − r Y j =1 (1 − q j ) ≥ 1 −   1 − 1 r r X j =1 q j   r , using the arithmetic-geom etric mean inequality . Finally , using co ncavity of φ r ( x ) = 1 − (1 − x r ) r and φ r (0) = 0 , we get 1 −   1 − 1 r r X j =1 q j   r = φ r   r X j =1 q j   ≥ φ r ( q − q s q t ) P r j =1 q j q − q s q t ≥ φ m − 1 ( q − q s q t ) P r j =1 q j q − q s q t = 1 q − q s q t 1 −  1 − q − q s q t m − 1  m − 1 ! r X j =1 q j , where the last ineq uality follows from the fact φ r ( x ) is dec reasing in r a nd by usin g r ≤ m − 1 . No tice that we u sed the fact P r j =1 q j ≤ q − q s q t for the ﬁrst ine quality in the reasoning above, which holds since r < t and th erefore P r j =1 q j ≤ q − q t ≤ q − q s q t . Case r ≥ t : As in th e p revious case we start by expan ding the left- hand side of 17. This tim e we bundle the tw o terms (1 − q s ) and (1 − q t ) when applying the arithmetic-geom etric mean ineq uality . r X j =1 q j j − 1 Y ℓ =1 (1 − q ℓ ) = 1 − r Y j =1 (1 − q j ) = 1 − (1 − q s )(1 − q t ) Y j ∈ [ r ] j / ∈{ s,t } (1 − q j ) ≥ 1 −   1 − 1 r − 1   − q s q t + r X j =1 q j     r − 1 . The remaining part of the proof is similar to the previous case. 1 −   1 − 1 r − 1   − q s q t + r X j =1 q j     r − 1 = φ r − 1   − q s q t + r X j =1 q j   ≥ φ r − 1 ( q − q s q t ) − q s q t + P r j =1 q j q − q s q t ≥ φ m − 1 ( q − q s q t ) − q s q t + P r j =1 q j q − q s q t = 1 q − q s q t 1 −  1 − q − q s q t m − 1  m − 1 !   − p s p t + r X j =1 q j   , again using co ncavity of φ r − 1 and the fact that φ r − 1 ( q − q s q t ) is d ecreasing in r . Th is ﬁnishes the pr oof of ( 17) and thus completes the proof of the lemma. Lev eraging Lemma B.2 we are now ready to prove Lemma 4 .10. W e recall th at f or a nonn egati ve su bmodu lar function f : 2 N → R + and p ∈ [0 , 1] N , its concave closure f + is deﬁned by f + ( p ) = max    X S ⊆ N α S f ( S )    α S ≥ 0 ∀ S ⊆ N , X S ⊆ N α S = 1 , X S ⊆ N ,i ∈ S α S = p i ∀ i ∈ N    . 38 Pr oo f of Lemma 4.10. Consider a basic solutio n ( α j , A j ) j ∈ [ m ] to the linear program th at deﬁnes f + ( p ) , i.e., f + ( p ) = P m j =1 α j f ( A j ) , with A j ⊆ N , α j ≥ 0 for j ∈ [ m ] , P m j =1 α j = 1 and P j ∈ [ m ] ,i ∈ A j α j = p i for i ∈ N . N otice th at since we chose a basic solution and th e LP deﬁning f + ( p ) only has n + 1 constraints apart from the nonnegati vity constraints, we ha ve m ≤ n + 1 . Let R ( b p ) be a random subset o f N co ntaining each elem ent i ∈ N indepen dently with probability bp i . W e distigu ish two cases depending on whether m ≤ n or m = n + 1 . Case m ≤ n : Consider the random set A = [ j ∈ [ m ] A j ( b · α j ) , where A j ( bα j ) is a random subset o f N co ntaining each element i ∈ N with proba bility b α j , indep endently of the others. Notice that the d istribution of A is dominated b y th e distribution of R ( b p ) since A contains each elemen t i ∈ N indepen dently with p robability Pr[ i ∈ A ] = 1 − Y j ∈ [ m ] i ∈ A j (1 − bα j ) ≤ 1 −     1 − X j ∈ [ m ] i ∈ A j bα j     = b p i = P r[ i ∈ R ( b p )] . Hence F ( b p ) ≥ E [ f ( A )] , and we can use the ﬁrst statement of Lemma B.2 to obtain F ( b p ) ≥ E [ f ( A )] ≥ 1 P m j =1 bα j 1 − 1 − P m j =1 bα j m ! m ! m X j =1 bα j f ( A j ) ≥  1 −  1 − b m  m  f + ( p ) , using P m j =1 α j = 1 and the fact tha t 1 − (1 − x m ) m x is decre asing in x . Th e proo f of this case is com pleted by o bserving that (1 − (1 − b m ) m ) is decreasing in m and m ≤ n . Case m = n + 1 : Since A j ⊆ N for j ∈ [ n + 1] and | N | = n , ther e mu st be at least one set A t that is covered by the remaining sets, i.e., A t ⊆ ∪ j ∈ [ n +1] ,j 6 = t A j . Furthermor e let s ∈ [ n + 1] \ { t } be the index m inimizing bα s . W e deﬁne prob abilities q j for j ∈ [ n + 1] as follows q j = ( bα j if j 6 = t, bα t 1 − bα s if j = t. W e follo w a similar approach as for the previous case b y considering the random set A = [ j ∈ [ m ] A j ( q j ) . Again, we ﬁrst observe that A is dominated by the d istribution of R ( b p ) . For any i ∈ N \ A t the analysis of the p revious case still holds an d sh ows Pr[ i ∈ A ] ≤ Pr[ i ∈ R ( b p )] . Consider now an element i ∈ A t . Let A t , A j 1 , . . . , A j r be all sets in th e family ( A j ) j ∈ [ n +1] that co ntain i . By our choice of A t , there is at least on e oth er set containing i , i.e., r ≥ 1 . Using (1 − q t )(1 − q j 1 ) =  1 − bα t 1 − b α s  (1 − bα j 1 ) α s ≤ α j 1 ≤ 1 − bα t − b α j 1 , we obtain Pr[ i ∈ A ] = 1 − (1 − q t )(1 − q j 1 ) r Y ℓ =2 (1 − q j ℓ ) = 1 − (1 − bα t − b α j 1 ) r Y ℓ =2 (1 − bα j ℓ ) ≤ 1 −     1 − X j ∈ [ n +1] i ∈ A j bα j     = b p i = Pr[ i ∈ R ( b p )] . 39 Therefo re, we ag ain have F ( b p ) ≥ E [ f ( A )] . Notice that q = P n +1 j =1 q j satisﬁes q = bα t − bα t 1 − b α s + n +1 X j =1 bα j = q s q t + n +1 X j =1 bα j = q s q t + b . W e apply the second statement of Lemma B.2 to the family ( A j ( q j )) j ∈ [ n +1] and use the above fact to obtain F ( b p ) ≥ E [ f ( A )] ≥ 1 q − q s q t  1 −  1 − q − q s q t n  n    − q s q t min { f ( A s ) , f ( A t ) } + n +1 X j =1 q j f ( A j )   = 1 b  1 −  1 − b n  n   − q s q t min { f ( A s ) , f ( A t ) } + ( q t − b α t ) f ( A t ) + bf + ( p )  ≥ 1 b  1 −  1 − b n  n   − q s q t f ( A t ) + ( q t − b α t ) f ( A t ) + bf + ( p )  . The claim follows by observing that q s q t = q t − b α t . C Details in constructing C R schemes via the ellipsoid algorithm Here we g iv e th e technical d etails that are inv olved in sampling and app roxima tely solving (DP1) an d ( LP1) from Section 4.2. First a short remind er of the primal and dual problem . (LP1) max c s.t. P φ ∈ Φ ∗ q i,φ λ φ ≥ x i c ∀ i ∈ N P φ ∈ Φ ∗ λ φ = 1 λ φ ≥ 0 ∀ φ ∈ Φ ∗ (DP1) min µ s.t. P i ∈ N q i,φ y i ≤ µ ∀ φ ∈ Φ ∗ P i ∈ N x i y i = 1 y i ≥ 0 ∀ i ∈ N W e start by observing that we can obtain strong estimates of q i,φ for any φ ∈ Φ ∗ . W e ass ume that φ is given as an oracle and can therefore be ev aluated in co nstant time. Proposition C.1 . Let φ ∈ Φ ∗ and i ∈ N . An estimate ˆ q i,φ of q i,φ whose err or is boun ded by ± ǫ x i with h igh pr o bability can be obtained in time polynomial in n an d 1 ǫ . Pr oo f. W e call a set S ⊆ N \ { i } good , if i ∈ φ ( S ∪ { i } ) . W e have q i,φ = P r[ i ∈ φ ( R )] = Pr[ i ∈ R and R \ { i } is goo d ] = P r[ i ∈ R ] · Pr[ R \ { i } is go od ] = x i · Pr[ R \ { i } is good ] . Notice that we can estimate Pr[ R \ { i } is good ] up to an error of ± ǫ with high probab ility b y a standard Monte Carlo approa ch, where we draw samples of R \ { i } . This can be d one in time polynomial in n and 1 ǫ and leads to t he claimed estimate ˆ q i,φ by the above fo rmula. W e n ow discuss how these estimates can be u sed to obtain a n ear-optimal solution to (DP1) by employing the ellipsoid method with a weak separ ation oracle. After th at we show ho w a near-optimal solution to (L P1) can be obtained. Notice that Proposition C.1 can e asily be used to obtain estimates e q i,φ of q i,φ that satisfy with h igh prob- ability e q i,φ ∈ [ q i,φ − ǫx i , q i,φ ] : it sufﬁces to consid er an estima te ˆ q i,φ of q i,φ that satisﬁes with high prob ability ˆ q i,φ ∈ [ q i,φ − ǫ 2 x i , q i,φ + ǫ 2 x i ] and to deﬁn e e q i,φ = ˆ q i,φ − ǫ 2 . In th e following, we assume that all used estimates e q i,φ satisfy e q i,φ ∈ [ q i,φ − ǫ x i , q i,φ ] . W e ca n obtain this with high pr obability through Prop osition C.1 since th e ellipsoid 40 that we will app ly in th e fo llowing only uses a poly nomial num ber of such estimates. Notice th at the se estimates are “pessimistic” estimates f or q i,φ , i.e., replacing q i,φ in (LP1) by the se estimates lead s to a lower o ptimal v alue o f the LP . Furthermo re, to simplify the expo sition, w e will assume that for any gi ven weigh t vector y ∈ R N + we can ﬁnd a C R scheme φ ∈ Φ ∗ maximizing P i ∈ N q i,φ y i . Th e following discussion works also if we can on ly ﬁnd a C R scheme φ th at appro ximately maximizes th is expression. T o app ly the ellipsoid algo rithm to (DP1) we d esign a weak separa tion oracle (see Chapter 4 in [32]). As a reminder, th e weak separation o racle has to provide the following guar antees. Gi ven is a non negative vector y = ( y i ) i ∈ N satisfying P i ∈ N x i y i = 1 , and a value µ . The weak separation oracle h as to provide eith er a feasible dual solution ( y ′ , µ ′ ) w ith µ ′ ≤ µ + ǫ , o r a hyperplan e separating ( y , µ ) from all feasible du al solutio ns. Giv en y and µ , let φ ∈ Φ ∗ be th e CR scheme that maximizes P i ∈ N q i,φ y i . If P i ∈ N e q i,φ y i ≤ µ , our w eak sep aration o racle retu rns the dual solution ( y , µ + ǫ ) . This solution ha s objectiv e value µ + ǫ as desired and is indeed feasible since for any φ ′ ∈ Φ ∗ , X i ∈ N q i,φ ′ y i ≤ X i ∈ N q i,φ y i ≤ X i ∈ N ( e q i,φ + ǫ x i ) y i = ǫ + X i ∈ N e q i,φ y i ≤ µ + ǫ. If P i ∈ N e q i,φ y i > µ , our weak separation oracle returns the separating hyperplane gi ven by the constrain t ( a , − 1) · ( z , ν ) ≤ 0 wh ere a is a n - dimension al vector with coef ﬁcients a i = e q i,φ for 1 ≤ i ≤ n (note th at that (DP1) has n + 1 variables correspon ding to y 1 , . . . , y n and µ ). First, this hyperp lane indeed cuts of f the solutio n ( y , µ ) . Fu rthermo re, if ( y ′ , µ ′ ) is a feasible dual solution then it satisﬁes the constraint since e q i,φ ≤ q i,φ : X i ∈ N e q i,φ y ′ i ≤ X i ∈ N q i,φ y ′ i ≤ µ ′ , where the secon d in equality in th e above follo ws from feasibility of ( y ′ , µ ′ ) . Hence, we obtained a weak s eparation oracle for (DP1), and the ellipsoid method can therefo re d etermine a feasible solution ( y , µ ) to (DP1) of value ≤ µ ∗ + 2 ǫ , where µ ∗ is the value of an optimal dual solution. Note that since ( y , µ ) is feasible, we have µ ∗ ≤ µ ≤ µ ∗ + 2 ǫ (see [32]). Let (DP1’) be the linear prog ram obtain ed from (DP1) b y only consider ing c onstraints co rrespon ding to C R schemes φ that were used in the ellipsoid algorithm while constructing the nearly optimal solution ( y , µ ) o f (DP1), which satisﬁes µ ≤ µ ∗ + 2 ǫ . Furthermore , we rep lace all te rms q i,φ by their estimates e q i,φ in ( DP1’). Hence, the feasible region of ( DP1’) co nsists of all separating h yperp lanes that wer e ge nerated d uring the ellipsoid algorithm. Notice that (DP1’) is a relaxation of (DP1) since our estimates satisfy e q i,φ ≤ q i,φ . Hence, the optimal value µ ′ of (DP1’) satisﬁes µ ′ ≤ µ ∗ . The e llipsoid algorithm actually certiﬁes th e ap prox imation quality of the generated solu tion ( µ, y ) by co mparing against the best solution satisfying the generated constraints, i.e., µ ≤ µ ′ + 2 ǫ. Let (LP1’) b e the d ual of (DP1’), and let ( λ ′ , c ′ ) be an optimal solution to (LP1’), which can be efﬁciently determin ed since (L P1’) ha s poly nomial size. W e retur n ( λ ′ , c ′ ) as the solution to (LP1). First, notice tha t ( λ ′ , c ′ ) is f easible f or (LP1) since e q i,φ ≤ q i,φ . Further more, c ′ = µ ′ ≥ µ − 2 ǫ ≥ µ ∗ − 2 ǫ = c ∗ − 2 ǫ, where the two equalities follow b y strong duality . 41

Submodular Function Maximization via the Multilinear Relaxation and Contention Resolution Schemes

Original Paper

Comments & Academic Discussion

Leave a Comment

Original Paper

Related Papers

Comments & Academic Discussion

Leave a Comment