Duality Gap, Computational Complexity and NP Completeness: A Survey

Computational Complexit y , NP Completeness and Optimization Dualit y: A Surv e y Prabh u Man y em Departmen t of Mathematics Shanghai Univ ersit y Shanghai 200444 , C hina. Email: prabhu.m anyem@gmail .com July 16, 2018 Abstract W e survey resear ch that studies the connectio n b etw een the computational complexity of optimization problems on the one hand, and the duality gap betw een the primal and dual optimization pro blems on the other . T o our kno wledg e , this is the first survey t hat connects the t wo very impo rtant areas. W e further lo ok a t a similar phenomenon in finite mo del theory r elating to complexity and optimization. Keyw ords . Computational complexit y , Optimizat ion, Mat hematical pr ogramming, Dualit y , Dualit y Gap, Finite model theo r y . AMS Classification . 90C25, 90C26, 90C46, 49N15 , 65K05, 68Q15, 68Q17, 68Q19 and 68Q25. 1 In tro duction In optimization problems, the duality gap is the difference b et we en the optimal solution v alues of the p r imal p roblem and the du al pr oblem. The relationship b et we en the du alit y gap and the computational complexit y of optimizatio n problems h as b een implicitly studied for the last few decades. Th e connection b etw een the t wo phenomena has b een su btly ac kn o wl- edged. The g ap h as b een exploited to design go o d approximat ion algorithms for NP-h ard optimization problems [2, 15, 24]. Ho wev er, w e ha ve b een unable to lo cate a single piece of literature that addresses this iss ue e xplicitly . This rep ort is an atte mpt to b ring a great deal of evidence together and sp ecifically a d dress this issu e. Do es the existence of p olynomial time algorithms for th e primal and the dual problems mean that the dualit y gap is zero? Conv ersely , do es the existence of a dualit y gap imply that either the primal problem or the dual problem is (or b oth are) NP-hard? Is there an in herent connection b et ween computational complexit y and str ong duality (that is, zero dualit y gap)? V ecte n and F asb ender (indep end en tly) we r e th e fir s t to disco ver the optimization dualit y [13]. They observ ed this phen omenon in the F ermat-T orric el li pr oblem : Giv en a triangle T 1 , find the equilateral triangle circumscrib ed outside T 1 with the maxim um height. They sho wed that th is maximum height H is equal to the minimum sum of the distances f rom th e v er tices to T 1 to the T orric el li p oint 1 . Thus, this p roblem enjoys strong d u alit y . The app aren t conn ection b et ween the dualit y gap and computational complexit y was con- sidered more than thirt y y ears ago . Linear Programming (LP) is a well kno wn optimiza tion problem. In the mid 1970 s, b efore Khac hiyan published his ellipsoid algorithm, LP was though t to b e p olynomially s olv able, precisely b ecause it ob eys str ong duality ; that is, the dualit y gap is zero. F or a go o d description of the ellipsoid algorithm, the reader is referr ed to the go o d b o ok by F ang and Puthenp u ra [9]. Strong duality also places the decision v ersion of Lin ear Programming in the class NP ∩ CoNP; see Lemma 9 below. W e should stress that t his is not a surv ey on Lagrangian dualit y or an y other f orm of opti- mization d ualit y . Rather, this is a sur v ey on the connections and relationships b et wee n the compu tational complexit y of optimization problems and dualit y . 2 Definitions A few definitions are p r o vided in this sec tion. Definition 1. ( D 1 ( r ) : Decision p roblem corre sp onding to a given minimization problem ) Giv en . An obje c tive function f ( x ) , as wel l as m 1 numb er of c onstr aints g ( x ) = b and m 2 numb er of c onstr aints h ( x ) ≥ c , wher e x ∈ R n is a ve ctor of var iables, b ∈ R m 1 and c ∈ R m 2 ar e c onstants. Al so given is a p ar ameter r ∈ R . T o Do . Determine if the set F = ∅ , wher e F = { x : g ( x ) = b , h ( x ) ≥ c and f ( x ) ≤ r } . (Ana lo gously, if the given p r oblem is maximization, then F = { x : g ( x ) = b , h ( x ) ≥ c and f ( x ) ≥ r } .) F is th e set of f easible solutions to the decisio n problem. Remark 2. A wor d of c aution: F or de cision pr oblems, the term “fe asibility” includes the c onstr aint on the obje ctive function; if the obje ctive function c onstr aint is violate d, the pr oblem b e c omes inf e asible. Definition 3. [3] ( Lagrangian Dual ) Supp ose we ar e given a minimization pr oblem P 1 such as Minimize f ( x ) : X → R ( X ⊆ R n ) , subje ct to g ( x ) = b , h ( x ) ≥ c , wher e x ∈ R n , b ∈ R m 1 and c ∈ R m 2 . (1) L et u ∈ R m 1 and v ∈ R m 2 b e two ve ctors of variables with v ≥ 0 . L et e = ( g , h ) . Assume that b = [ b 1 b 2 · · · b m 1 ] T and c = [ c 1 c 2 · · · c m 2 ] T ar e c olumn ve ctors. The fe asible r e gion for the primal pr oblem i s X . 1 The T orricelli p oint X is ind eed the one with the least sum of the distances | AX | + | BX | + | C X | from the vertices A , B and C of T 1 . 2 F or a given primal pr oblem as in P 1 , the L agr angian dual pr oblem P 2 is define d as fol lows: Maximize θ ( u , v ) subje ct to v ≥ 0 , wher e θ ( u , v ) = inf x ∈ R n { f ( x ) + m 1 X i =1 u i ( g i ( x ) − b i ) + m 2 X j =1 v j ( h j ( x ) − c i ) } . (2) Note that g i ( x ) − b i = 0 [ h j ( x ) − c j ≥ 0] is t he i th equalit y [ j th inequalit y] constrain t. 3 Bac kground: Dualit y and the classes NP and CoNP (Note to Review ers : Hop e the definitions and explanat ion in this section is sufficient . If not, please let us kno w where we should exp an d and elab orate.) W e n ow tur n our atten tion to the relationship b et ween the dualit y of an optimizati on problem, and mem b ersh ip in the complexit y classes NP and CoNP of the corresp onding set of decision problems. D ecision p roblems are th ose with y es/no an s w ers , as opp osed to optimizatio n problems th at retur n an optimal solution (if a f easible solution exists). Corresp on d ing to P 1 defined ab o ve in (1), t here is a set D 1 of decision problems, defined as D 1 = { D 1 ( r ) | r ∈ R } . The defi nition of D 1 ( r ) w as provided in Def. 1. Let us no w d efine th e computational classes NP , CoNP and P . F or more details, the in terested reader is referred to either [2] or [2 1]. W e b egin with th e follo w ing w ell kn o wn definition: Definition 4. NP (r e sp e ctively P ) is the c lass of de cision pr oblems for which ther e exist non- deterministic (r esp e ctively deterministic) T uring machines which pr ovide Y es/No answers in time that is p olynom ial in the size of the input instanc e. In p articular, for pr oblems in P and NP, if the answer is yes , th e T uring machine (TM) is able to pr ovide an “evidenc e ” (in te chnic al terms c al le d a certificate ), such as a fe asible solution to a g iven instanc e. The class CoNP of de cision pr oblems is similar to N P, exc ept for one key differ enc e: the TM is able to pr ovide a c ertific ate only for no answ e rs. F r om the ab ove, it fol lows that f or an instanc e of a pr oblem in NP ∩ CoNP, the c orr esp onding T uring machine c an pr ovide a c ertific ate for b oth yes and no instanc es. F or example, if D 1 ( r ) in De f. 1 ab o ve is in NP , the certifica te will b e a feasible s olution; that is, an x ∈ R n whic h ob eys the constrain ts g ( x ) = b , h ( x ) ≤ c an d f ( x ) ≥ r. (3) On the other hand, if D 1 ( r ) ∈ CoNP , the certificate will b e an x ∈ R n that viol ates at least one of the m 1 + m 2 + 1 constrain ts in ( 3). Remark 5. F or pr oblems in NP, for Y es instanc es, extr acting a sol ution fr om the c ertific ate is not always an e fficient (p olynomial time) task. Similarly, in th e c ase o f Co NP, pinp ointing a violation fr om a T uring machine c ertific ate 2 is not guar ante e d to b e efficient either. 2 W e th ank W enXun Xing and PingKe Li of Tsinghua Universit y ( Beijing) for p onting ou t t he above . 3 Remark 6. P ⊆ NP , b e c ause any c omputation that c an b e c arrie d out by a deterministic TM c an also b e c arrie d out by a non-deterministic TM. The pr oblems in P ar e de cidable deterministic al ly in p olynomial time. The class P is the same as its c omplement Co-P. That is, P is close d u nder c omplementation . F urthermor e, Co-P ( ≡ P ) is a subset of CoNP. We know that P is a subset of N P. Henc e P ⊆ NP ∩ CoNP . Thus for an instanc e of a pr oblem in P, the c orr e sp onding T uring machine c an pr ovide a c ertific ate for b oth yes and no instanc e s. W e are no w ready to define what is m ean t by a tight dual , and ho w it relates to the in tersection class of problems, NP ∩ CoNP . Note that for tw o problems to b e tight duals, it is su fficien t if they are tight with resp ect to just one t yp e of dualit y (suc h as Lagrangian d u alit y , for example). Definition 7. Tight dual s and the class TD . Two optimiza tion pr oblems P a and P b ar e dual to e ach other if the dual of one p r oblem is the other. Supp ose P a and P b ar e dual to e ach other, with zer o duality gap; then we say that P a and P b ar e tight duals . F or any r ∈ R , let D a ( r ) and D b ( r ) b e the de cision versions of P a and P b r e sp e ctively. L et TD b e the class of al l de cision pr oblems whose optimization versions have tight duals. That is, TD is th e set of al l pr oblems D a ( r ) and D b ( r ) for any r ∈ R . Remark 8. A wor d of c aution. Tight duality is not the same as str ong duality. F or str ong duality, it is sufficient if ther e exist fe asible solutions to the primal and th e d ual such th at th e duality gap is zer o. F or tight duality to hold, we also r e quir e that the primal pr oblem P a and the dual pr oblem P b b e dual to o ne another. One wa y in whic h dualit y gaps are related to the classes NP and CoNP is as fol lo ws: Lemma 9. [21] TD ⊆ NP ∩ CoNP. F rom Remark 6 and Lemma 9, we know th at b oth TD and P are su bsets of NP ∩ CoNP . But is there a con tainment r elationship b et ween TD and P? That is, is either TD ⊆ P or P ⊆ TD? This is the s ub ject of further stu dy in this pap er, with p articular reference to Lagrangian dualit y . Remark 10. We should mention that in sever al c ases, given a primal pr oblem P , even if we ar e able to find a dual pr oblem D such tha t the dual of P is D , it do e s not ne c essarily fol low that the dual of D is P . That is,the dual of the dual ne e d not b e the primal. P and D ar e not ne c essarily duals of e ach other. We do not include such ( P , D ) p airs in TD . A mong primal-dual p airs of pr oblems, TD is a r estricte d class. 4 Lac k of S trong Dualit y results in NP hardness In this sectio n , w e will review results from the li terature, whic h sh o w that the lack of strong dualit y imply that the optimization p roblem in question is NP-hard , assu ming that the primal problem ob eys the c onstr aint qualific ation assumption as stated b elow in Def. 14 . Here we 4 w ork w ith Lagrangian dualit y . Results for other t yp es of dualit y such as F enc h el, geometric and canonical dualities require further in vestig ation. Let us define wh at we mean b y wea k dualit y (as opp osed to tigh t dualit y and strong dualit y): Definition 11. Given a prima l pr oblem P 1 and a dual pr oblem P 2 , a s define d in Def. 3 , the p air ( P 1 , P 2 ) is said to ob ey w e a k duality if θ ( u , v ) ≤ f ( x ) , for eve ry fe asible solution x to the primal and every fe asible solution ( u, v ) to the dual . Definition 12. In Def. (11), if (i) the ine q uality is r eplac e d by an e quality, and (ii) ther e exist a primal fe asible solution ¯ x and a dual fe asible solution ( ¯ u , ¯ v ) such that the e quality in (i) h olds, then the p air ( P 1 , P 2 ) is said to b e ob ey str ong duality . The follo win g theorem from [3] guaran tees that the feasible solutions to Lagrangian dual problems (1) and (2) ind eed ob ey we ak duality : Theorem 13. If x is a fe asible so lu tion to the primal pr oblem in (1) a nd ( u , v ) i s a fe asible solution to the dual pr oblem in (2), then f ( x ) ≥ θ ( u , v ) . W e shall no w d efine a sp ecial t yp e of con ve x program, called a c onvex pr o gr am with c onstr aint qualific ation , whic h is on e with an assump tion ab out the existence of a feasible solution in the interior of the domain. Definition 14. Convex pro gram (convex optimization p roblem) . Giv en . A c onvex set X ⊂ R n , two c onvex functions f ( x ) : R n → R and g ( x ) : R n → R m 1 , as wel l as an affine function h ( x ) : R n → R m 2 . T o do . Minimize f ( x ) , subje ct to g ( x ) ≤ 0 , h ( x ) = 0 and x ∈ X . Definition 15. Convex pro gram with c onstraint q u a lification . Same a s the optimization pr oblem in Def. 14, exc ept th at we include th e fol lowing constrain t qualification assum ption . Ther e is an x 0 ∈ X such that g ( x 0 ) < 0 , h ( x 0 ) = 0 , and 0 ∈ int h ( X ) , wher e h ( X ) = \ x ∈ X h ( x ) . ( Note : Of course, the fu nctions ab o ve can b e written in the same form as in Def. 1 and 2. In suc h a case, we can define ( g ( x ) − b ) to b e a con v ex function and ( h ( x ) − c ) to b e an affine function, wh ere b ∈ R m 1 and c ∈ R m 2 .) Definition 16. In Def. 14, if any of the ( m + 1 ) functions f ( x ) : R n → R and g ( x ) : R n → R m 1 is not c onvex, then the optimization pr oblem is said to non-c onvex . F or the remainder of this section, w e w ill assume pr imal constr aint qualification; that is, w e assume that constrain t qualification is applied to the pr imal optimization prob lem. T h e follo wing theorem pr o vides sufficien t c onditions under whic h strong dualit y can o ccur: Theorem 17. Str ong Duality [3]. If (i) the primal pr oblem is giv e n as in Def. 14, and (ii) the primal and dual pr oblems have fe asible solutions, then the primal and dual optima l 5 solution values ar e e qual (that is, the duality gap is zer o): inf { f ( x ) : x ∈ X , g ( x ) ≤ 0 , h ( x ) = 0 } = sup { θ ( u , v ) : v ≥ 0 } , θ ( u , v ) = in f x ∈ X { f ( x ) + m 1 X i =1 u i g i ( x ) + m 2 X j =1 v j h j ( x ) } , (4) wher e θ ( u , v ) is the dual obje ctive function. Using th e co n tr ap ositiv e statemen t of Th eorem 17, w e get the follo wing result: Corollary 18. (to The or em 17) If ther e exists a duality gap using L agr angian duals, then either the primal or the dual is not a c onvex optimization pr oblem. (R ememb er, we ar e assuming c onstr aint qu alific ation.) The Subset Sum p roblem is defined as follo ws: Given a set S of p ositiv e integ er s { d 1 , d 2 , · · · , d k } and another p ositiv e in teger d 0 , is there a su bset P of S , suc h that the sum of the integ ers in P equals d 0 ? Using a p olynomial time red uction from the Subset Sum problem to a n on-con vex optimization problem (see Def. 16), Murt y a n d K abadi (1987) sho w ed the follo wing: Theorem 19. [19] If an op timization pr oblem is non-c onvex, it is NP-har d. In certain cases, non-conv ex p roblems ha ve an equiv alen t con vex formulat ion, for example, through strong du alit y . S uc h a du al transformation, where a conv ex pr oblem B is a dual of a non-con v ex problem A suc h that the dualit y gap b et ween them is zero, is call ed hidden convexit y [4, 5]. In suc h cases, the reform u lated con ve x problem is also NP-hard ; otherwise the p rimal non-conv ex pr oblem can b e solv ed efficien tly , th us viola ting Theorem 19. Definition 20. S tandard Quadratic Program (SQP) [6]. Minimize the function x T Q x , wher e x ∈ ∆ (the standa r d simplex i n R n + ). The n ve rtic es of ∆ ar e at a unit distanc e (in the p ositive dir e ction) along e ach of the n axes of R n . Q is a given symmetric matrix in R n × n . The con v erse of Th eorem 19 is not true. A con vex optimization problem in general is NP-hard, SQP b eing an example. S QP is non-con ve x ; how ev er in [6 ], Bomze a nd de Clerk p ro ve that it has an exac t conv ex reformulat ion as a c op ositive pr o gr amming pr oblem. SQP is kno wn to b e NP-hard, since its d ecision version contai ns the max-clique problem in graphs as a sp ecial case. F rom this, it follo w s that copositive programming is also NP-hard; see [7] for more on this topic. F rom Corollary 18 and Th eorem 19, it follo ws that Theorem 21. Assuming c onstr aint qualific ation, if ther e exists a duality gap using L a- gr angian duals, then either the pr i mal or the dual is NP-har d. These results are tru e for Lagrangian d ualit y . F or other types of dualit y su ch as F enchel, geometric and canonica l dualities, this requires further in vestig ation. 6 5 Do es Strong Dualit y Imp ly P olynomial Time Solv abilit y? A t this time, su c h a p ro of (of wh ether a dualit y gap of zero implies p olynomial time solv abilit y of th e primal and the dual pr ob lems) app ears p ossible only for v ery simple problems, since estimating the dualit y gap app ears extremely c hallenging for many problems. Some of the problems where this is true include C on vex Programming (and in particular, Linear Programming), where b oth the primal and the dual problems are con vex; the p olyno- mial time alg orithms are deriv ed from interior p oin t method s . F or e xample, see the b o ok by Nestero v and Nemiro vs kii [20]. In a recent pap er [17], we ha ve d emonstrated an additional computational b enefit arising from strong d ualit y . Primal-dual problem pairs that are p olynomially solv able and ob ey strong dualit y can b e solv ed b y a sin gle call to a de cisi on T uring machine , that is, a T ur ing mac hine that pr o vides a Y es/No ans w er (if the ans w er is yes, then it can p ro vid e a feasible solution which s u pp orts the Y es answer). Previously , it was only kn o wn that optimization problems require m ultiple calls to a decision T uring machine (for example, doing a binary searc h on the solution v alue to obtain an optimal solution). F or more details, the reader is referred to [17]. More inv estigatio n is needed to answer the question Do e s Str ong Duality Imply Polynomial Time Solvability? in a general setting. Ho we v er, there ha ve b een some results recently using Canonical du alit y for certain typ es of quadratic p rograms [10, 25] . Consider a standard quadratic programming (primal) p r oblem: Minimize P ( x ) = 1 2 x T A x + b T 0 x sub j ect to b T i x ≤ c i , 1 ≤ i ≤ m, (5) where A = A T ∈ R n × n , b i ∈ R n for 1 ≤ i ≤ m , and x ∈ R n . A is a symmetric matrix. W e can wr ite the Lag r angian f u nction as L ( x , λ ) = 1 2 x T A x + b 0 + m X i =1 λ i b i ! T x − m X i =1 λ i c i , λ i ≥ 0 . (6) The fi rst order necessary condition among the Karush-Ku hn-T uck er (KKT) conditions yields A x + b 0 + m X i =1 λ i b i = 0, from whic h we get the v alue of x a s x = − A − 1 b 0 + m X i =1 λ i b i ! , (7) assuming that A is an inv ertible matrix. S ubstituting th is v alue of x bac k into the Lagrangian function yields Q ( λ ) = − 1 2 b 0 + m X i =1 λ i b i ! T A − 1 b 0 + m X i =1 λ i b i ! − m X i =1 λ i c i . (8) 7 W e then g et a dual problem, the canonical dual P d , as follo w s : Maximize Q ( λ ) , sub j ect to A ≻ 0 a nd λ ≥ 0 . (9) (The relation A ≻ 0 mea ns the matrix A should b e p ositiv e definite.) The p ositiv e definiteness of A has b een found to b e a su fficien t condition. Whether it is a necessary cond ition is not kno wn yet. No w, Q ( λ ) is a conca v e function, to b e maximized in the dual v ariable λ ; hence P d can b e solv ed efficien tly , since we are maximizing a conca ve f unction. Thus in general, w e ca n get a lo we r b ound for the p rimal problem qu ic kly; and in some cases, we can get a strong dual with zero dualit y gap, which provides an optimal solution for the primal problem in p olynomial time. Canonical dualit y was first dev elop ed to a d dress the problem of (p ossibly large) dualit y gaps in Lagrangian d u alit y in the con text of problems in analytical mec hanics [11]. F urth ermore, large dualit y gaps can also b e found while solving non-con vex optimizatio n problems usin g F enc hel-Morro w-Ro ck afellar dualit y . The so-called c anonic al dual tr ansformatio n can b e used to form ulate a strong-dual problem (that is, one with a zero dualit y gap). Quoting from [11], the primal pr oblem c an b e made e qu al to its c anonic al dual in the sense that they have the same KKT p oints . 5.1 Preliminary Results: Canonical Dualit y and the Complexit y Classes NP and CoNP W e consider quadratic programming 3 problems with a single quadratic constrain t (QCQ P) [25]. T he pr imal problem P 0 is giv en by: Minimize P ( x ) = 1 2 x T A x − f T x sub j ect to 1 2 x T B x ≤ µ, (10) where A and B are non-zero n × n symm etric mat rices, f ∈ R n , and µ ∈ R . ( A , B , f and µ are give n .) Corresp onding to the primal P 0 , the ca nonical dual problem P d is as follo w s: sup σ P d ( σ ) = − 1 2 f T ( A + σ B ) − 1 f − µσ sub j ect to σ ∈ F = { σ ≥ 0 | A + σ B ≻ 0 } , (11) assuming that F is non-empty . T o ensur e that ( A + σ B ) is in ve rtible, we ha ve assumed that A and B are symmetric and non-zero. Again, the p ositiv e defi niteness of ( A + σ B ) for the dual pr oblem has been found to b e a sufficien t co n dition, a n d it is not kno wn yet whether it is a necessa ry co ndition. The follo w ing theorem app eared in [25]: 3 Thanks to Shu-Cherng F ang (NCSU, USA) for his inpu t here. 8 Theorem 22. (Str ong duality the or em) When the maximum value P d ( σ ∗ ) in (11) is finite, str ong duality b etwe en th e primal pr oblem P 0 and the dua l pr oblem P d holds, and the optimal solution for P 0 is given by x ∗ = ( A + σ ∗ B ) − 1 f . Ho we v er, the pair ( P 0 , P d ) ma y not b e tigh t d uals (i.e., they ma y not b elong to the class T D ). T o sho w that ( P 0 , P d ) ∈ T D , it also needs to b e sho wn that the dual of P d is P 0 . (Recall Remark 10.) Remark 23. We have not pr ovide d a detaile d analysis of how the c anonic al pr oblem c an b e derive d. The inter este d r e ader is r eferr e d to [25]. Our g o al is to il lustr ate an instanc e of a p air of prima l and d ual pr oblems that o b ey str ong duality, a nd wher e an op timal solution c an b e obtaine d effic i ently (in p olynomial time). Strong du alit y for a v ariation of QC Q P w as established b y Mor ´ e and Sorensen in 1983 [18] (as describ ed in [1]); th ey call theirs as the T rust Region Problem (TRP). The pr ob lem is to minimize a quadratic function q 0 , sub ject to a n orm constrain t x T x ≤ δ 2 . This is an example of a n on-con vex problem w h ere strong d ualit y holds. R en dl and W olk owic z [23] s ho w that the TRP can b e reformulat ed as an u nconstrained conca v e maximization problem, and hence can b e solv ed in p olynomial time b y using the inte rior point metho d [20]. Recall that decision problems are those w ith Y es/No answ ers. Combining Theorem 22 with Lemma 9 abov e, we can co nclude that Remark 24. De c ision versions of Pr oblems P 0 in (10) and P d in (11) ar e memb e rs of the interse ction class NP ∩ CoNP. F urthermore, from Theorem 3 in [25], it is easy to see that problems P 0 and P d can b e solv ed in p olynomial time. The authors in [25] u s e what they call a “b ou n darification” tec hnique whic h mo v es the analytical solution ¯ x to a glo b al minimizer x ∗ on the b oundary of the pr imal domain. This strengthens the conjecture th at Conjecture 25. If two optimizatio n pr oblems P a and P b ar e such that one of them is the dual of the ot her, that is, they exhibit str ong duality, then b oth ar e p olynomial ly solvable. As hin ted earlier, the canonical dual feasible space F in (11) ma y b e emp t y . In suc h a case, strong dualit y still h olds on a feasible domain in whic h the matrix ( A + σ B ) is n ot definite. Ho wev er, the question as to ho w to solv e the canonical dual problem is still op en. It is conjectured in [12] that the p r imal problem (from which the canonical dual problem is deriv ed ) could b e NP-hard. The observ ations ab o ve are f or quadratic programming problems with a single quadratic constrain t. It w ould b e in teresting to see what happ ens f or quadratic programming p roblems with t wo constraints, whether strong dualit y still holds, and wh ether b oth the prim al and the dual are still p olynomially solv able. Semidefinite Pr o gr amming (SDP) . Ramana [22] exhib ited strong du alit y for the SDP pr oblem. Ho we v er, the complexit y of SDP is u nknown; it w as sh o wn in [22] that the decision version of S DP is NP-complete if and o nly if NP = CoNP . (Note : There ha v e b een some pub lished pap ers which sa y that SDP is p olynomial time solv able. T his is NOT correct, as the ab ov e result in [22] sh o ws.) 9 6 Descriptiv e Complexit y and Fixed P oin ts On a final note, w e would like to briefly describ e a s imilar phenomenon whic h o ccurs in the field of Descriptiv e Complexity , whic h is the app lication of Fi nite Mo d el theory to computational complexit y . In particular, we would like t o men tion least fixed p oint (LFP) co m p utation. A full description would b e b ey ond the scop e of this p ap er. Ho wev er, we w ould like to briefly men tion a few rela ted concepts and phenomena. F or a go o d description of least fixed p oint s (LFP) in existen tial second order (ES O ) logic, th e reader is referr ed to [14] (c hapters 2 and 3) and [8]. If the inp ut structures are o rdered, then expressions in LFP logic ca n describ e p olynomial time (PTIME) computation [14]. The input instance to an LFP computation consists of a structur e A , wh ic h includ es a domain set A and a set of (fir st order) relations R i , eac h with arity r i , 1 ≤ i ≤ J . The LFP computation works b y a stagewise add ition of tup les from A , to a n ew relation P (of some arit y k ). If P i represent s the relatio n (set of tuples) after stage i , then P i ⊆ P i +1 . The transition from P i to P i +1 is th rough an op erator Φ, such that P i +1 = Φ( P i ). A t the b eginning, P is empt y , that is, P 0 = ∅ . F or some v alue of i , sa y when i = f , if P f = P f +1 , a fixe d p oint has b een rea c hed. Without going into details, let us just sa y that suc h a fixed p oin t, reac hed as ab o ve, is also a le ast fixed p oin t (LFP) if the op erator Φ can b e chosen in a particular manner. The in terested reader is referred to [14] (c hapter 2) for details. Note that the num b er of elements in P can b e at most | A | k (where | A | is the n u mb er of elemen ts in A ), wh ic h is p olynomial in the size of the domain. Hence f ≤ | A | k , so an LFP is ac hiev ed within a p olynomial num b er of stag es. Similar to LFP , w e can also d efine a gr e atest fi xed p oin t (GFP). This is obtained b y doing the reverse; we start w ith the entire set A k of k -ary tuples fr om the univ er s e A , and then remo ving tuples fr om P in stages. A t the b eginning, P 0 = A k . In fu r ther stages, P i ⊃ P i +1 . The GFP is reac hed at stage g if P g = P g +1 . The logic that includes LFP and GFP expressions is known as LFP logic . It expresses decision problems (those with a Y es/No answer), such as th ose in Def. 1 and 4. T o b e feasible, a solution should also ob ey th e ob jectiv e function constraint ( f ( x ) ≥ K or f ( x ) ≤ K ). The LFP computation expresses d ecision problems based on maximization. Before the fi xed p oint is reac hed, the solution is in feasible; that is, the num b er of tuples in the fixed p oin t relation P is insufficien t. Ho wev er, once the fixed p oint is reac hed, the solution b ecomes feasible. Similarly , the GFP computation expresses decision problems based on minimization. Problem . An in teresting problem arising in L FP Logic is this: F or what t yp e of primal-dual optimization pr oblem pairs will the LFP and GFP computation m eet at the same fix ed p oin t? Do es th is mean that suc h a pair is p olynomially solv able? 7 Conclusion and F ur ther Study Let u s again stress that this is not a sur v ey on Lagrangian du alit y or any other form of optimization du alit y . Rather, this is a survey on the connections and relationships b et we en the compu tational complexit y of optimization problems and dualit y . 10 In this pap er, w e ha ve touc hed the tip of the iceb erg on a v ery interesting problem, that of connecting the computational hardn ess of an optimization problem w ith its dualit y c harac- teristics. A lot more study is required in th is area. Another issue is that of sadd le p oint for Lagrangian duals. Th is is a decidable problem; we can d o bru te force and find t he primal and dual optimal solutions; this will tell u s if there is a du alit y ga p. If the ga p is ze ro, then there is a saddle p oin t. (Jeroslo w [16] sho wed that the intege r programming problem w ith quadratic constrain ts is undecidable if the num b er of v ariables is unb ounded, wh ich is an extreme condition. Ho wev er, if eac h v ariable has a fi nite up p er and lo w er b oun d, then the num b er of solutions is finite and th u s it is p ossible to determine the best sol ution in finite time.) Ho we v er, this problem w ould b e NP-complete, un less w e can tell whether it has a saddle p oin t b y looking at the s tructure of the problem o r by run ning a p olynomial time alg orithm. W e hop e that this pap er will motiv ate further researc h in this very interesti ng topic. 8 Ac kno wledgmen ts This researc h w as supp orted b y (i) a visiting fello w ship from the National Cheng Kung Uni- v ersity (NCKU), T ainan, T aiwa n ; (ii) the S hanghai Leading Academic Discipline Pr o ject #S30104; and (iii) the National Natural Science F oundation of Chin a (# 11071158) . Sup- p ort from all sources is gratefully ac kn o wledged. References [1] An streic her, K. and W olk o wicz, H. On Lagrangian relaxation of qu adratic matrix con- strain ts. SIAM Journal on Matrix Analysis and A pplic ations , 22(1), 41–55 (electronic) (2000 ). [2] Au siello, G., Crescenzi, P ., Gambosi, G., Kann, V., Marc h etti-Spaccamel a, A. and Pro- tasi, M. Combina torial Optimization Pr oblems and Their A ppr oximability Pr op erties . Springer, Germany (1999). [3] Bazaraa, M.S ., Sherali, H.D. and S h ett y , C.M. N on-Line ar Pr o gr amming: The ory and Algor ithms (thir d e dition) . Wiley In terscience (2 006). [4] Bec k, A. Conv exit y Prop erties Asso ciated with Noncon ve x Q u adratic Matrix F un c- tions and Applications to Quadratic Programming. Journal o f Optimization The ory a nd Applic ations , 142, 1–29 (200 9). [5] Ben-T al, A. and T eb oulle, M. Hidden con v exit y in some noncon v ex quadratically con- strained quad r atic programming. Mathemalic al Pr o gr amming , 72, 51–63 (1 996). [6] Bomze, I. M. and d e Klerk, E. S olving Standard Qu adratic Optimizati on Problems Via Linear, Semidefinite and Cop ositiv e Pr ogramming. Journal of Glob al Optimization , 24, 163–1 85 (20 02). 11 [7] D ¨ ur , M. Cop ositive Pr o gr amming - a Survey . in Diehl, M., Glineur , F., J arlebr ing, E. and Mic hiels, W. (eds.): R e c ent A dvanc es i n Optimization and its Applic ations in Engine ering , pp. 3–20, Springer (2010). [8] E b binghaus, H.D. and Flum, J. Finite Mo del The ory . S p ringer (1999). [9] F ang, S.C. and Pu thenpur a, S. Line ar Optimization and Extensions: The ory and Algo- rithms . P ren tice Hall , New Jersey (1993). [10] Gao, D.Y. Canonical dualit y theory and solutio ns to constr ained noncon vex quadr atic programming. Journal of Glob al Optimization , 29(4), 377–3 99 (2004). [11] Gao, D.Y., Ruan, N. and S herali, H.D. Solutions and optimalit y criteria for noncon- v ex constrained global optimizat ion problems with conn ections b etw een canonical and Lagrangian du alit y . Journal of Glob al O ptimization , 45, 473– 497 (2009). [12] Gao, D.Y. Solutions and optimalit y criteria to b o x constrained noncon vex minimization problems. Journal of Ind ustrial and Management Optimization , 3 (2), 293–304 (20 07). [13] Giannessi, F. O n the theory of Lagrangian dualit y. Optimization L etters , 1, 9–20 (2007). [14] Gr¨ adel, E., Kolaitis, P .G., Libkin, L., Marx, M., S p encer, J., V ardi, M.Y., V enema, Y. an d W einstein, S. Finite Mo del The ory and Its Applic ations . T exts in Th eoretical Computer Science. Springer (2007). [15] Ho ch baum, D. (ed.). Appr oximation Algorithms for NP- H ar d P r oblems . PWS Pub lishing Compan y , Boston, M A (1997). [16] Jeroslow, R.G. There C an n ot b e an y Algorithm for Integ er P rogramming with Quadratic Constrain ts. O p e r ations R ese ar ch , 21, 221–2 24 (1973). [17] Many em, P . Synta ctic expressions to exp r ess NP-hard optimizatio n problems and p rob- lems with zero dualit y gap, 2011. T o app ear in the journal Optimization .. [18] Mor´ e , J.J. and Soren sen, D.C. Computing a trust region step. SIAM Journal on Scientific and Statistic al Computing , 4(3), 553–572 (1 983). [19] Mur t y , K.G. and Kabadi, S.N. Some NP-complete p roblems in qu adratic and non lin ear programming. M athematic al Pr o gr amming , 39, 117–129 (1987). [20] Nestero v, Y. and Nemiro vskii, A. Interior Point Polynomial Metho ds in Convex Pr o- gr amming: The ory and A pplic ations . S o ciet y for In d ustrial and App lied Mathematic s , Philadelphia (1994). [21] Papadimitriou, C.H. Computational Complexity . Addison-W esley , Reading, Mas- sac husetts (1994). [22] Ramana, M.V. An Exact Dualit y Theory for Semidefinite P r ogramming and Its Com- plexit y Implications. M athematic al Pr o gr amming , 77, 129–16 2 (1997 ). [23] Rend l, F. and W olk owicz , H. A semidefinite f ramew ork for tru st region subproblems with applications to large s cale minimization. Mathematic al Pr o gr amming , 77(2, Series B), 273–299 (19 97). 12 [24] V azirani, V.V. Appr oximation Algorith ms . Sprin ger, Germany (2001). [25] Xing, W., F ang, S-C., Gao, D.Y., S h eu, R.L. and Zhang, L. Canonical Dual Solutions to the Quadratic P rogramming o ver a Quadr atic C onstrain t. T ec hnical rep ort, Department of Mathematical Sciences, Tsinghua Univ ersity , Beijing, Ch ina (2008). 13