On the linear independence of spikes and sines

ON THE LINEAR INDEPENDENCE OF SPIKES AND SINES JOEL A. TROPP Abstra ct. The purp ose of this work is to survey what is kn own ab out th e linear ind epend en ce of spike s an d sines. The paper provides new results for the case where the locations of th e spikes and the frequen cies of the sines are c hosen at random. This problem is equiv alen t to studying the sp ectral norm of a random submatrix drawn from the discrete F ourier transform matri x. The pro of dep ends on an extrapolation argument of Bourgain and Tzafriri. 1. Introduction An in vestig ation central to sparse approxima tion is whether a giv en col lection of impu lses an d complex exp onen tials is linearly indep enden t. T his inquiry app ears in the early pap er of Donoho and Stark on uncertaint y principles [DS89], and it h as b een rep eated and amplified in the w ork of subsequent authors. In d eed, researc hers in sparse approximat ion hav e dev elop ed a muc h deep er understand ing of general dictionaries b y pr obing the structur e of the unassumin g dictionary that con tains only spikes and sin es. The purp ose of this w ork is to survey wh at is kno wn ab out the linear in dep endence of spik es and sines and to pro vid e some new results on random sub collectio ns chosen fr om this dictionary . The m etho d is adapted from a pap er of Bourgain–Tzafriri [BT91]. The adv antag e of this approac h is that it a vo ids some of the complicated com bin atorial argumen ts that are u s ed in relat ed w orks, e.g., [CR T06]. The pro of also applies to other t yp es of dictionaries, although w e do not pur sue this line of inqu ir y here. 1.1. Spik es and Sines. Let us sh ift to formal discussion. W e work in th e inn er-pro duct sp ace C n , and we use the symbol ∗ for the conjugate transp ose. Define th e Hermitian inn er pro duct h x , y i = y ∗ x and th e ℓ 2 v ecto r norm k x k = |h x , x i| 1 / 2 . W e also write k·k for the sp ectral n orm, i.e., the op erator norm for linear maps from ( C n , ℓ 2 ) to itself. W e consider tw o orthonorm al bases for C n . Th e standard b asis { e j : j = 1 , 2 , . . . , n } is giv en by e j ( t ) = ( 1 , t = j 0 , t 6 = j for t = 1 , 2 , . . . , n. W e often refer to the elemen ts of the standard basis as spikes or impulses . The F ourier basis { f j : j = 1 , 2 , . . . , n } is giv en b y f j ( t ) = 1 √ n e 2 π i j t/n for t = 1 , 2 , . . . , n. W e often refer to the elemen ts of the F our ier basis as sines or c omplex exp onentia ls . The discr ete F ourier tr ansform (DFT) is the n × n matrix F whose ro ws are f ∗ 1 , f ∗ 2 , . . . , f ∗ n . The matrix F is unitary . In particular, its sp ectral norm k F k = 1. Moreo ver, the en tries of the DFT matrix are boun ded in magnitude b y n − 1 / 2 . Let T and Ω b e subsets of { 1 , 2 , . . . , n } . W e write Date : 4 S eptem b er 2 007. Revised 15 April 2008. 2000 M athemat ics Subje ct Classific ation. Pri mary: 46B07, 47A11, 15A52. S econdary: 41A46. Key wor ds and phr ases. F ourier analysis, lo cal theory , rand om matrix, sparse appro ximation, un certain t y principle. The author is with App lied & Computational Mathematics, MC 217-50 , Cali fornia Institute of T e chnolo gy , 1200 E. Califo rnia Blvd., P asadena, CA 91125-500 0. E-mail: jtro pp@acm.cal tech.edu . Supp orted b y NSF 050329 9. 1 2 JOEL A. TROPP F Ω T for the r estrictio n of F to the ro ws listed in Ω and the column s listed in T . Since F Ω T is a submatrix of the DFT matrix, its sp ectral n orm do es not exceed one. W e us e th e analysts’ co n v en tion that upright letters represen t un iv ersal constants. W e reserv e c for small constants and C for large constants. The v alue of a constant ma y c hange at eac h app earance. 1.2. Linear Indep endence. Let T and Ω b e su bsets of { 1 , 2 , . . . , n } . Consider the collec tion of spik es and sines listed in these sets: X = X ( T , Ω) = { e j : j ∈ T } ∪ { f j : j ∈ Ω } . T o day , w e will discu s s metho ds for determining when X is linearly indep endent. Sin ce a lin early indep endent collection in C n con tains at most n vec tors, w e obtain a simple necessary condition | T | + | Ω | ≤ n . Dev eloping su fficien t conditions, how ev er, requires more sophistication. W e ap p roac h the problem by studying the Gram matrix G = G ( X ), whose en tries are the inner pro ducts b et w een pairs of elemen ts from X . It is ea sy to c heck that the Gram matrix can b e expressed as G =  I | Ω | F Ω T ( F Ω T ) ∗ I | T |  where I m denotes an m × m identi t y matrix and |·| denotes the cardin alit y of a set. It is well kno wn that the coll ection X is linearly indep endent if and on ly if its Gram m atrix is nonsingular. The Gram matrix is n onsingular if and only if its eigen v alues are nonzero. A basic (and easily confirmed ) fact of matrix analysis is th at the extreme eigen v alues of G are 1 ± k F Ω T k . Therefore, the c ol le ction X is line arly i ndep endent if and only if k F Ω T k < 1. One may also attempt to quan tify the exten t to w h ic h col lection X is linearly indep endent. T o t hat end , define the c ond ition numb er κ of the Gram matrix, whic h is the ratio of its largest eigen v alue to its smallest eigen v alue: κ ( G ) = 1 + k F Ω T k 1 − k F Ω T k . If k F Ω T k is b ounded a w ay from one, then the condition num b er is c onstan t. One ma y in terp ret this statemen t as evidence the collectio n X is strongly linearly indep end en t. The r eason is that the condition n umb er is the recipro cal of the relativ e sp ectral-norm distance b etw een G an d the nearest singular matrix [Dem97 , p. 33 ]. As w e ha ve men tioned, G is singular if and only if X is linearly dep endent. This article f ocuses on s tatement s ab out lin ear in dep endence, rather than conditioning. Nev er- theless, m an y resu lts can b e adapted to obtain precise inf orm atio n ab out the size of k F Ω T k . 1.3. Summary of Results. Th e ma jor result of this pap er to show that a random coll ection of spik es and s in es is extremely likely to b e strongly linearly indep endent, provided that the total n um b er of spikes and sines do es not exceed a constan t prop ortion of the ambien t dimen sion. W e also provi de a result whic h shows that the norm of a prop erly scaled random sub matrix of the DFT is at most constan t with high p robabilit y . F or a more detailed statemen t of these theorems, turn to S ecti on 2.3. 1.4. Outline. The next section pr o vides a surv ey of b ounds on the norm of a submatrix of the DFT matrix. It concludes with d etailed new results for the case where the submatrix is random. Section 3 con tains a pro of of the new results. Nu m erical exp eriments are presen ted in S ectio n 4, and Section 5 describ es some add itional researc h directions. Ap p endix A contai ns a pro of of the k ey backg round result. SPIKES A N D SINES 3 2. Histor y and Resul ts The strange, ev en tfu l history of our problem can b e viewe d as a sequence of b ound s on norm of the matrix F Ω T . Resu lts in th e literature can b e divided into t wo classes: the case wh ere the sets Ω and T are fixed and the case where one of the sets is ran d om. In th is w ork , we in vestiga te wh at happ ens wh en b oth sets are c hosen r andomly . 2.1. Bounds for fixed sets. An early result, d ue to Donoho and Stark [DS89], asserts that an arbitr ary collec tion of spik es and sines is linearly indep endent, provided that the collection is not to o big. Theorem 1 (Donoho–Stark) . Supp ose that | T | | Ω | < n . Then k F Ω T k < 1 . The original argumen t relie s on the fact that F is a V andermonde matrix. W e presen t a short pro of that is completely analytic. A similar argument u sing an inequalit y of Sch ur yields the more general resu lt of Elad and Bruckstein [EB02, Thm. 1]. Pr o of. Th e entries of the | Ω | × | T | matrix F Ω T are uniform ly b ounded by n − 1 / 2 . Since the F rob enius norm dominates the sp ectral norm, k F Ω T k 2 ≤ k F Ω T k 2 F ≤ | Ω | | T | /n . Under the hyp othesis of the theorem, th is quant it y do es not exceed one.  Theorem 1 has an eleg an t corollary th at follo ws im m ediatel y from the basic inequalit y for geo- metric and arithmetic means. Corollary 2 (Donoho–Sta rk) . Supp ose that | T | + | Ω | < 2 √ n . Then k F Ω T k < 1 . The cont rap ositiv e of Theorem 1 is us ually int erpreted as an discr ete unc ertainty principle : a v ecto r and its discrete F ourier transform cannot simultaneously b e sparse. T o expr ess this cla im quan titativ ely , we define the ℓ 0 “quasinorm” of a ve ctor by k α k 0 = | { j : α j 6 = 0 }| . Corollary 3 (Donoho–Stark) . Fix a ve ctor x ∈ C n . Consider the r epr esentations of x in the standar d b asis and the F ourier b asis: x = X n j =1 α j e j and x = X n j =1 β j f j . Then k α k 0 k β k 0 ≥ n . The example of the Dir ac c omb sho ws th at Theorem 1 and its corollaries are sharp . Supp ose that n is a s q u are, and let T = Ω = { √ n, 2 √ n, 3 √ n, . . . , n } . On account of the P oisson sum matio n form ula, X j ∈ T e j = X j ∈ Ω f j . Therefore, the set of v ectors X ( T , Ω) is linearly d ep enden t and | T | | Ω | = n . The su bstance b ehind this example is that the ab elian group Z / Z n con tains non trivial subgroups when n is comp osite. T he presence of these su bgroups leads to arithm etic cancelatio ns for prop erly c h osen T and Ω. See [DS89] for add itional discu ssion. One w a y to eradicate the cancelation ph enomenon is to require that n b e prime. In this case, the group Z / Z n has n o non trivial subgroup . As a r esult, m uc h larger collect ions of spik es and sines are lin early indep end en t. Compare the follo win g r esult with Corollary 2. Theorem 4 (T ao [T ao 05, Thm. 1.1]) . Supp ose tha t n is prime. If | T | + | Ω | ≤ n , then k F Ω T k < 1 . The p roof of Th eorem 4 is algebraic in nature, and it do es not pro vide inf ormation ab out con- ditioning. Indeed, one exp ects that some sub matrices ha v e norms v ery near to one. When n is co mp osite, sub grou p s of Z / Z n exist, bu t they hav e a v ery r igid structure. Conse- quen tly , one c an also a void cancelations b y c ho osing T and Ω with care. In particular, one ma y consider the situation where T is clustered and Ω is spread out. Do noho and L oga n [DL92] study 4 JOEL A. TROPP this case usin g th e analytic principle of the lar ge sieve , a p o werful tec hnique from num b er theory that can b e traced bac k to the 1930 s. See the lecture n otes [Jam06] for an engaging in tro du ctio n and r eferences. Here, we simp ly restate the (sharp) large siev e inequalit y [J am06, L S 1.1] in a manner that exp oses its connection with our problem. The spr e ad of a set is measur ed as the difference (mo dulo n ) b etw een the closest pair of ind ices. F o rmally , define spread(Ω) = min {| j − k mo d n | : j, k ∈ Ω , j 6 = k } with the conv ent ion th at the mo dulus returns v alues in the sym metric range {−⌈ n/ 2 ⌉ +1 , . . . , ⌊ n/ 2 ⌋} . Observe that | Ω | · spread(Ω) ≤ n . Theorem 5 (Large Siev e Inequalit y) . Supp ose that T is a blo ck of adjac ent indic es: T = { m + 1 , m + 2 , . . . , m + | T |} for an inte ger m. (2.1) F o r e ach set Ω , we have k F Ω T k 2 ≤ | T | + n/ sp read(Ω) − 1 n . In p articular, when T has form (2.1) , the b ound | T | + n / sp read(Ω) < n + 1 implies that k F Ω T k < 1 . Of course, we can reverse the roles of T and Ω in this th eorem on accoun t of d ualit y . T h e same observ ation applies to other results where the t w o s ets do not participate in the same wa y . The discussion ab o ve shows that there are cases where delicately constructed sets T and Ω lead to linea rly dep endent collectio ns of spikes and sines. Explicit conditions that rule out the bad examples are unkno wn, b ut n ev ertheless the bad examples turn out to b e qu ite rare. T o quan tify this intuition, we must in tro duce pr ob ab ility . 2.2. Bounds when one set is random. In their w ork [DS89, Sec. 7.3], Donoho and Stark d iscu ss n umerical exp erimen ts designed to study what happ ens when on e of the set s of spik es or sines is dra wn at random. Th ey conject ure that the situation is v astly d ifferen t from th e ca se wh ere the spik es and sines are chosen in an arbitrary fashion. Within the last few ye ars, researc h ers h a ve made substanti al theoretical p rogress on this question. Indeed, we will see that the linea rly depen d en t collect ions form a v anish ing p r op ortion of all collections, pro vid ed that the total n umb er of spikes and s ines is slightly sm alle r than the dimension n of the v ector s p ace. First, we describ e a probabilit y mo d el for rand om sets. Fix a num b er m ≤ n , and consider the class S m of ind ex sets that ha v e card inalit y m : S m = { S : S ⊂ { 1 , 2 , . . . , n } and | S | = m } . W e ma y construct a random set Ω b y d ra win g an element from S m uniformly at random. That is, P { Ω = S } = | S m | − 1 for eac h S ∈ S m . In th e sequ el, we su bstitute the symb ol | Ω | for the letter m , and we say “Ω is a r andom set with cardinalit y | Ω | ” to describ e th is t yp e of random v ariable. This phr ase should cause no confusion, and it allo ws us to av oid extra notation for the cardinalit y . In the sparse appro ximation literature, the first rigorous result on rand om sets is due to Cand` es and Romb erg. T h ey study the case where one of the sets is arbitrary and the other set is c hosen at r an d om. Their pro of dr a ws hea vily on th eir pr ior w ork with T a o [CR T06]. Theorem 6 (Cand` es–Rom b erg [CR06, Thm. 3.2]) . Fix a numb er s ≥ 1 . Supp ose that | T | + | Ω | ≤ c n p ( s + 1) log n . (2.2) If T is an arbitr ary set with c ar dinality | T | and Ω is a r andom se t with c ar dinal ity | Ω | , then P n k F Ω T k 2 ≥ 0 . 5 o ≤ C(( s + 1) log n ) 1 / 2 n − s . SPIKES A N D SINES 5 The nu meric al c onsta nt c ≥ 0 . 2791 , pr ovide d that n ≥ 512 . One should in terpret this theorem as follo ws. Fix a set T , and consid er all sets Ω that satisfy (2.2) . Of these, th e p rop ortion that are not strongly linearly in dep enden t is only ab out n − s . On e should b e a ware that th e logarithmic factor in (2.2) is intrinsic when one of the sets is arbitrary . Indeed, one can constr u ct examples related t o the Dirac com b whic h show that the failure probabilit y is constan t unless the logarithmic factor is present. W e omit the details. The pro of of Th eorem 6 ultimately inv olv es a v ariation of the momen t metho d for stu dying random matrices, which was initiated by Wigner. T he k ey p oin t of the argumen t is a b ound on the exp ected trace of a high p o w er of the r andom matrix p n/ | Ω | · F ∗ Ω T F Ω T − I | T | . Th e calculations in v olve delicate combinatoria l tec hniques that dep end hea vily on the structure of the matrix F . This approac h can also b e used to establish that the smallest singular v alue of F Ω T is b ounded w ell a wa y from zero [CR T06, Thm. 2.2]. Th is low er b oun d is essen tial in many app lications, but w e do not need it here. F or extensions of these ideas, see also the w ork of Rauhut [Rau07]. Another result, similar to T heorem 6 , suggests that the arbitrary set and the random set do n ot con tr ib ute equally to th e sp ectral n orm. W e present one v ersion, whose deriv ation is adapted from [T ro07, Th m. 10 et seq.]. Theorem 7. Fix a numb er s ≥ 1 . Supp ose that | T | log n + | Ω | ≤ c n s . If T is an arbitr ary set of c ar dinality | T | and Ω is a r ando m set of c ar dinality | Ω | , then P n k F Ω T k 2 ≥ 0 . 5 o ≤ n − s . The p roof of this theorem u ses Rud elson’s selection lemma [Rud 99, Sec. 2] in an essenti al w ay . This lemma in turn hinges on the n oncomm utativ e Khintc hine inqualit y [LP86, Buc01]. F o r a related app licat ion of this approac h, see [CR07]. Theorems 6 and 7 are in teresting, but they d o not pr edict that a far more striking phenomenon o ccurs. A random collection of sin es has the follo wing prop ert y with high probabilit y . T o this collect ion, one can add an arbitr ary set of spikes without sacrificing linear indep endence. Theorem 8. Fix a numb e r s ≥ 1 , and as sume n ≥ N ( s ) . Exc ept with pr ob ability n − s , a r andom set Ω whose c ar dinal ity | Ω | ≤ n/ 3 has the f ol lowing pr op erty. F or e ach set T whose c ar dinality | T | ≤ c n s log 5 n , it ho lds that k F Ω T k 2 ≤ 0 . 5 . This result follo ws f r om the (deep) fact that a r andom r o w-sub matrix of th e DFT matrix satisfies the r estricte d isometry pr op erty (RIP ) with high probabilit y . More precisely , a r andom set Ω with cardinalit y | Ω | ve rifies the follo w ing condition, exce pt with probabilit y n − s . | Ω | 2 n ≤ k F Ω T k 2 ≤ 3 | Ω | 2 n when | T | ≤ c | Ω | s log 5 n . (2.3) This result is adapted from [R V06, Th m. 2.2 et seq.]. The b ound (2.3) was originally established by Cand ` es and T ao [CT06] for s ets T wh ose cardinalit y | T | ≤ c | Ω | /s log 6 n . Rudelson and V ers h yn in deve lop ed a simpler pro of and reduced the exp onent on the logarithm [R V06]. Exp erts believ e that the correct exp onen t is just one or t w o, b ut this conjecture is presen tly out of r eac h. Pr o of. Let c b e the constan t in (2.3). Abbreviate m = c | Ω | /s log 5 n , and assu me that m ≥ 1 for no w. Dra w a rand om set Ω with cardinalit y | Ω | , so relation (2.3) h olds except with probability n − s . Select an arbitrary set T whose cardinalit y | T | ≤ c n/ 6 s log 5 n . W e ma y assume that 2 | T | /m ≥ 1 6 JOEL A. TROPP b ecause | Ω | ≤ n/ 3. P artition T into at most 2 | T | /m disjoin t b locks, eac h con taining no more than m ind ices: T = T 1 ∪ T 2 ∪ · · · ∪ T 2 | T | /m . Apply (2.3) to calculate that k F Ω T k 2 ≤ 2 | T | m max k k F Ω T k k 2 ≤ | T | · 2 s log 5 n c | Ω | · 3 | Ω | 2 n ≤ 1 2 . Adjusting constan ts, we obtain the result when | Ω | is n ot to o s mal. In case m < 1, dr a w a random set Ω and then draw additional random co ordinates to form a larger set Ω ′ for wh ic h c | Ω ′ | /s log 5 n ≥ 1 and | Ω ′ | ≤ n/ 3. This c hoice is p ossible b ecause n ≥ N ( s ). Apply the foregoing argument to Ω ′ . S ince th e s p ectral n orm of a su bmatrix is not larger than the norm of th e enti re matrix, we ha v e the b oun d k F Ω T k 2 ≤ k F Ω ′ T k 2 ≤ 0 . 5 for eac h sufficien tly small set T .  2.3. Bounds when b oth sets are random. T o mo v e into the regime where the num b er of spik es and sin es is prop ortional to the dimension n , we need to randomize b oth sets. The ma jor goal of this article is to establish the follo win g th eorem. Theorem 9. Fix a numb er ε > 0 , and assume that n ≥ N ( ε ) . Supp ose that | T | + | Ω | ≤ c( ε ) · n. L et T a nd Ω b e r ando m sets with c ar dinalities | T | and | Ω | . Then P n k F Ω T k 2 ≥ 0 . 5 o ≤ exp  − n 1 / 2 − ε  . The c onstan t c( ε ) ≥ e − C /ε . Note th at the pr obabilit y b ound here is sup er p olynomial, in contrast with the p olynomial b ounds of the pr evious s ecti on. The estimate is essen tially optimal. T ake ε > 0, and s u pp ose it w ere p ossible to obtain a b oun d of th e form P {k F Ω T k = 1 } ≤ exp {− n 1 / 2+ ε } where | T | + | Ω | ≤ 2 n 1 / 2 . According to Stirling’s app ro ximation, there are ab out exp { n 1 / 2 log n } w a ys to select t w o sets sat- isfying the cardin alit y b ound . A t the same time, the prop ortion of sets that are linearly dep end en t is at most exp {− n 1 / 2+ ε } . Multiplying these t wo qu an tities, w e fi nd that no pair of s ets meeting the cardinalit y boun d is linearly dep endent. Th is claim cont radicts the fact that the Dirac comb yields a linearly dep end en t collectio n of size 2 n 1 / 2 . Remark 10. As we wil l se e, The or em 9 holds f or every n × n matrix A with c onstant sp e c tr al norm and u niformly b ounde d entries: k A k ≤ 1 and | a ω t | ≤ n − 1 / 2 for ω , t = 1 , 2 , . . . , n . The pr o of do es not r ely on any sp e c i al pr op erties of the discr ete F o urier tr ansform. 2.4. Random matrix theory. Finall y , we consider an application of this approac h to random matrix theory . Not e that eac h column of F Ω T has ℓ 2 norm p | Ω | /n . T herefore, it is appropriate to rescale the matrix by p n/ | Ω | so that its columns ha v e un it n orm. Under this scaling, it is p ossible that the norm of the m atrix explo d es wh en | Ω | is small in comparison with n . The con tent of the next result is that this ev en t is highly unlikely if the submatrix is d ra wn at random. Theorem 11. Fix a numb er δ ∈ (0 , c) . Supp o se that n ≥ N ( δ ) and tha t | T | ≤ | Ω | = δ n. If T and Ω ar e r andom sets with c ar dinalities | T | and | Ω | , then P  r n | Ω | k F Ω T k ≥ 9  ≤ n − C . SPIKES A N D SINES 7 F or δ in the range [c , 1], it is eviden t that r n | Ω | k F Ω T k ≤ c − 1 . Therefore, we obtain a constant b ound for the n orm of a normalized r andom subm atrix throughout the en tire parameter range. Remark 12. The or em 11 also holds for the class of matric es describ e d in R emark 10. 3. Norms of random subma trices In this section, we pro v e Theorem 9 and Th eorem 11. Firs t, we describ e some problem s implifi- cations. Then w e p ro vid e a moment estimate for the norm of a very small random su bmatrix, and w e p resen t a device for extrap olating a momen t estima te for the norm of a m uc h larger random submatrix. Th is momen t estimat e is used to prov e a tail boun d, which quic kly le ads to the tw o ma jor results of the pap er. 3.1. Reductions. Denote by P δ a random n × n diagonal matrix wh er e exactly m = ⌊ δ n ⌋ entries equal on e and the r est equal zero. This matrix can b e seen as a pr o jector onto a random set of m co ordinates. With this n otat ion, the restriction of a matrix A to m r andom rows and m random columns can b e expressed as P δ AP ′ δ , w here the t w o pro jectors are statistically indep end en t fr om eac h other. Lemma 13 (Square case) . L et A b e an n × n matrix. Supp ose tha t T and Ω ar e r ando m sets with c ar dinalities | T | and | Ω | . If δ ≥ max {| T | , | Ω |} /n , then P {k A Ω T k ≥ u } ≤ P    P δ AP ′ δ   ≥ u  for u ≥ 0 . Pr o of. It s u ffices to s ho w that the prob ab ility is weakly increasing as the cardinalit y of one set increases. Th erefore, w e f ocus on Ω and remo v e T from the notation for clarit y . Let Ω b e a random subset of cardinalit y | Ω | . Cond itional on Ω , w e may dra w a uniformly r andom elemen t ω from Ω c , and p ut Ω ′ = Ω ∪ { ω } . This Ω ′ is a un if orm ly random su bset with cardinalit y | Ω | + 1. W e hav e P {k A Ω k ≥ u } = E I ( k A Ω k ≥ u ) ≤ E I (   A Ω ∪{ ω }   ≥ u ) = E I ( k A Ω ′ k ≥ u ) = P {k A Ω ′ k ≥ u } where we hav e wr itten I ( E ) for the ind icato r v ariable of an ev en t. The inequalit y follo ws b ecause the sp ectral norm is wea kly in creasing w h en we pass to a larger m atrix, and so we h a ve the in clusion of ev ents { Ω ′ : k A Ω k ≥ u } ⊂ { Ω ′ :   A Ω ∪{ ω }   ≥ u } .  It can b e incon v enien t to w ork w ith pro jectors of th e form P δ b ecause their entries are d ep enden t. W e w ould pr efer a mo del where co ordinates are selected indep endently . T o that en d , denote b y R δ a random n × n diagonal matrix wh ose entries are indep endent 0–1 rand om v ariables o f mean δ . This matrix can b e seen as a pro jector onto a rand om s et of co ordinates w ith aver age cardinality δ n . Th e f ollo wing lemma establishes a relationship b et w een the t w o t yp es of co ordinate pr o jectors. The argum en t is drawn from [CR06, S ec. 3]. Lemma 14 (Random co ordinate mo dels) . Fix a numb er δ in [0 , 1] . F or ev ery n × n matrix A , P {k P δ A k ≥ u } ≤ 2 P {k R δ A k ≥ u } for u ≥ 0 . In p ar ticular, P    P δ AP ′ δ   ≥ u  ≤ 4 P    R δ AR ′ δ   ≥ u  for u ≥ 0 . 8 JOEL A. TROPP Pr o of. Giv en a co ord inate pro jector R , d enote b y σ ( R ) the s et of coord inates on to whic h it p r o jects. F or typographical f elicit y , w e use # σ ( R ) to in dicate the cardinalit y of this set. First, supp ose that δ n is an intege r. F or eve ry u ≥ 0, w e ma y calculate that P {k R δ A k ≥ u } ≥ X n j = δ n P {k R δ A k ≥ u | # σ ( R δ ) = j } · P { # σ ( R δ ) = j } ≥ P {k R δ A k ≥ u | # σ ( R δ ) = δ n } · X n j = δ n P { # σ ( R δ ) = j } ≥ 1 2 P {k P δ A k ≥ u } . The second inequalit y holds b ecause the s p ectral norm of a su b matrix is smaller than the sp ectral norm of the matrix. The third inequalit y relies on the fact [JS68, Thm. 3.2] that the medians of the binomial distrib ution binomial ( δ, n ) lie b et ween δ n − 1 and δ n . In case δ n is not integral , th e monotonicit y of the sp ectral norm yields that P {k R δ A k ≥ u } ≥ P    R ⌊ δn ⌋ /n A   ≥ u  . Since P ⌊ δn ⌋ /n = P δ , this p oint completes the argumen t.  3.2. Small submatrices. W e fo cus on matrices with u n iformly b ounded entrie s. The first step in the argumen t is an elemen tary estimate on the n orm of a random submatrix with exp ected order one. In this regime , the b ound on the matrix entries determines the n orm o f the su bmatrix; the signs of the en tries do not pla y a role. The pro of shows that most of the v ariation in the norm actually deriv es fr om the fl uctuation in the ord er of the s ubmatrix. Lemma 15 (Small Su bmatrices) . L et A b e an n × n matrix whose entries ar e b ounde d in magnitude by n − 1 / 2 . Abbr eviate  = 1 /n . When q ≥ 2 log n ≥ e ,  E   R  AR ′    2 q  1 / 2 q ≤ 2 q n − 1 / 2 . Pr o of. By homogeneit y , we may resca le A s o that its en tries are b ound ed in mag nitude b y on e. Define the eve n t Σ j k where th e random submatrix has order j × k . Σ j k = { # σ ( R  ) = j and # σ ( R ′  ) = k } . On this even t, the n orm of th e sub m atrix can b e b ounded as   R  AR ′    ≤   R  AR ′    F ≤ p j k . Using elemen tary in equalitie s, we ma y estimate the p robabilit y that this ev en t occur s . P (Σ j k ) =  n j  n k   j + k (1 −  ) 2 n − ( j + k ) ≤  e n j  j  e n k  k n − ( j + k ) = (e /j ) j · (e /k ) k . With this inf orm atio n at hand, the rest of the pro of follo ws from some easy calculations: E   R  AR ′    2 q = X n j,k =1 E h   R  AR ′    2 q | Σ j k i · P (Σ j k ) ≤ X n j,k =1 ( j k ) q · (e /j ) j · (e /k ) k = h X n k =1 k q · (e /k ) k i 2 . A short exercise in differen tial calculus sh o ws that the maxim u m term in the sum occur s when k log k = q . W rite k ⋆ for th e solution to this equat ion, and note that k ⋆ ≤ q . Bounding all the terms by the maxim um, we fi nd X n k =1 k q · (e /k ) k ≤ n · exp { q log k ⋆ − k ⋆ log k ⋆ + k ⋆ } ≤ n · exp { q log k ⋆ } ≤ n · q q . SPIKES A N D SINES 9 Com bining the last t w o inequalities, w e reac h  E   R  AR ′    2 q  1 / 2 q ≤  n 2 · q 2 q  1 / 2 q = n 1 /q · q . When q ≥ 2 log n , the fi rst term is less than t w o.  Remark 16. Th is ar gument delivers a moment estimate that is r oughl y a factor of log q smal ler than the one state d. This f act c an b e use d to sharp e n the major r esults slightly at a c ost we pr efer to avoid. 3.3. Extrap olation. The k ey tec hnique in the pro of is an extrap olatio n of the momen ts of the norm of a large random submatrix from the momen ts of a smalle r random s u bmatrix. Without additional information, extrap olation must b e fruitless b ecause the signs of matrix en tries pla y a critical r ole in determinin g th e sp ectral norm . It turns out that we can fold in in f ormation ab out the signs b y incorp orating a b oun d on the sp ectral norm of the matrix. The pr oof, whic h we p ro vide in App endix A, ultimately dep ends on the minimax p rop ert y of the Chebyshev p olynomials. The metho d is essen tially the same as the one Bourgain and Tzafriri develo p to p r o ve Prop osition 2.7 in [BT91]. See also [T ro08, S ec. 7]. Prop osition 17. Supp ose that A is a n n × n matrix with k A k ≤ 1 . L et q b e an inte ger that satisfies 13 log n ≤ q ≤ n/ 2 . Write  = 1 /n , and cho ose δ in the r ange [1 /n, 1] . F or e ach λ ∈ (0 , 1) , it ho lds that  E   R δ AR ′ δ   2 q  1 / 2 q ≤ 8 δ λ max  1 , n λ  E   R  AR ′    2 q  1 / 2 q  . Although th e statemen t is a little complicat ed, we require the fu ll p o wer of this estimate. As usual, the parameter q is the momen t that w e seek. T h e prop osition extrap olates f rom a matrix of exp ected order 1 up to a matrix of exp ecte d order δn . T h e parameter λ is a tuning kn ob that con tr ols ho w muc h of the estimate is d etermined by the sp ectral norm of the f ull matrix and ho w m uc h is determined b y the norm bou n d for sm all s ubmatrices. Indeed, the first mem b er of the maxim um reflects the sp ectral norm b ound k A k ≤ 1. 3.4. A tail b ound. W e are n o w prepared to dev elop a tail b ound for the random norm k R δ AR ′ δ k . Lemma 18 (T ail Bound) . L et A b e an n × n matrix for which k A k ≤ 1 and | a j k | ≤ n − 1 / 2 for j, k = 1 , 2 , . . . , n . Cho ose δ fr om [1 /n, 1] and an inte ger q that sat isfies 13 log n ≤ q ≤ n/ 2 . F or e ach λ ∈ (0 , 1) , it holds that P n   R δ AR ′ δ   ≥ 8 δ λ max  1 , 2 q n λ − 1 / 2  · u o ≤ u − 2 q for u ≥ 1 . Pr o of of L emma 18. Cho ose an in teger q in the range [13 log n, n/ 2]. Mark ov’s inequalit y allo ws that P    R δ AR ′ δ   ≥  E   R δ AR ′ δ   2 q  1 / 2 q · u  ≤ u − 2 q . Therefore, we m ay establish th e result by obtaining a momen t estimate. This estimate is a direct consequence of L emm a 15 and Prop osition 17:  E   R δ AR ′ δ   2 q  1 / 2 q ≤ 8 δ λ max n 1 , n λ · 2 q n − 1 / 2 o . Com bine the tw o b oun ds to complete the argumen t.  The t w o ma jor results of this pap er, Theorem 9 and Theorem 11, b oth follo w from a simp le corollary of Lemma 18. 10 JOEL A. TROPP Corollary 19. Supp ose that T and Ω ar e r andom sets with c ar dinalities | T | and | Ω | . Assume δ ≥ max {| T | , | Ω |} /n . F o r e ach inte ger q tha t satisfies 13 log n ≤ q ≤ n/ 2 and for λ ∈ [0 , 1] , it holds that P n k F Ω T k ≥ 8 δ λ max  1 , 2 q n λ − 1 / 2  · u o ≤ 4 u − 2 q for u ≥ 1 . Pr o of. Consid er th e matrix A = F . P erform the red uctions from Section 3.1, Lemma 13 and Lemma 14. Then app ly the tail b ound , Lemma 18.  3.5. Pro of of Theorem 9. The con ten t of Theorem 9 is to p ro vide a b ound on δ which ensures that k F Ω T k is somewhat le ss than one w ith extremely high p robabilit y . T o that end, we wan t to mak e λ close to zero and q large. T h e follo win g selections accomplish this go al: λ = log 16 log(1 /δ ) and q = ⌊ 0 . 5 n 1 / 2 − λ ⌋ . Note that w e can make λ as small as w e like by taking δ sufficien tly sm all. F or an y v alue of λ < 0 . 5, the n um b er q satisfies th e requirement s of Corollary 19 as so on as n is sufficient ly large. No w, th e b ound of Corollary 19 results in P {k F Ω T k ≥ 0 . 5 u } ≤ 4 u − 2 q . F or u = √ 2, we see that P n k F Ω T k 2 ≥ 0 . 5 o ≤ 4 · 2 − q . If f ollo ws that, for an y assignable ε > 0, w e can make P n k F Ω T k 2 ≥ 0 . 5 o ≤ exp  − n 1 / 2 − ε  pro vided that δ ≤ e − C /ε = c( ε ) and that n ≥ N ( ε ). 3.6. Pro of of T heorem 11. T o establish Theorem 11, w e must make the parameter λ as close to 0 . 5 as p ossible. Cho ose λ = 1 2 − 0 . 1 log(1 /δ ) and q = ⌊ C log n ⌋ . where C is a large constant. These c hoices are acceptable once δ is sufficien tly sm all and n is sufficien tly large. Corollary 19 delive rs P n k F Ω T k ≥ 8 . 9 δ 1 / 2 u o ≤ 4 u − C log n . F or u = 90 / 89, w e r eac h P n k F Ω T k ≥ 9 δ 1 / 2 o ≤ n − C , adjusting constan ts as necessary . Finally , we tr an s fer the factor δ 1 / 2 to the other side of the inequalit y and set δ = | Ω | /n to complete th e pr o of. 4. Numerical Experiments The th eorems of this pap er p ro vide gross information ab out the norm of a random submatrix of the DFT. T o complement these results, w e p erformed some numerical exp erimen ts to giv e a more detailed emp ir ical view. The first set of experiments concerns random squ are submatrices of a DFT matrix of size n , where w e v aried th e p arameter n o v er sev eral o rders of magnitude. Giv en a v alue of δ ∈ (0 , 0 . 5), w e formed one hundred random sub matrices with dimensions δ n × δ n and computed the av erage sp ectral norm of these matrices. W e did not plot data when δ ∈ (0 . 5 , 1) b ecause the norm of a random submatrix equals one. SPIKES A N D SINES 11 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Proportion of rows/cols ( δ ) Expected norm Norm of random square submatrix drawn from n × n DFT Conjectured limit n = 1024 n = 128 n = 40 Figure 1. S ample a v erage of the norm of a random δ n × δ n subm atrix d ra wn from the n × n DFT. Figure 1 shows th e ra w data f or this first exp erimen t. As n gro w s, one ca n see that the norm tends tow ard an apparen t limit: 2 p δ (1 − δ ) . In Figure 2, we re-scale eac h matrix b y δ − 1 / 2 so its columns hav e un it n orm and then compute th e a v erage sp ectral norm. More elab orate b ehavior is visible in this plot: • F or δ = 1 /n , the norm of a random submatrix is id en tically equal to one. • F or δ = 2 /n , th e norm tends to w ard 1 + 2 − 1 / 2 = 1 . 7071 . . . , whic h can b e v erified b y a relativ ely s imple analytic computation. • The maximum v alue of th e norm app ears to o ccur at δ = 2 / √ n . • The app aren t limit of the scaled n orm is 2 √ 1 − δ , in agreemen t with the first fi gu r e. These phenomena are intrig uing, and it would b e v aluable to understand th em in more detail. Unfortunately , the metho ds of this p ap er are n ot refi n ed enough to p ro vid e an explanation. In the second set of exp eriments, w e s tu died the norm of a random r ecta ngular submatrix of the 128 × 128 DFT matrix. W e v aried the p rop ortion δ T of co lumns and the prop ortion δ Ω of ro ws in the range (0 , 1). F or eac h p air ( δ T , δ Ω ), we drew 100 random submatrices and compu ted the a verage n orm. Figure 3 shows the raw data. The apparent trend is that E   P δ Ω F P ′ δ T   = 2 p δ (1 − δ ) where δ = | T | + | Ω | 2 . Figure 4 shows the same data, rescale d b y max {| T | , | Ω |} − 1 / 2 . As in the square case , th is plot rev eals a v ariet y of int eresting phenomena that are worth atten tion. 5. Fur ther Research Directions The present researc h suggests sev eral d irections for futur e exp loratio n. (1) It ma y b e p ossible to impro v e the constant s in Pr op ositio n 17 u sing a v ariatio n of the current approac h. Instead of using the Chebyshev p olynomial to estimate the coefficients of th e p olynomial that arises in the pro of, one might u se the n onnegativ e p olynomial of least 12 JOEL A. TROPP 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 Proportion of rows/cols ( δ ) Expected norm * δ −1/2 Scaled norm of random square submatrix drawn from n × n DFT Conjectured limit n = 1024 n = 256 n = 128 n = 80 n = 40 Figure 2. S ample a v erage of the norm of a random δ n × δ n subm atrix d ra wn from the n × n DFT and r e-scal ed b y δ − 1 / 2 . 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 1.2 Proportion of columns ( δ T ) Norm of random rectangular submatrix drawn from 128 × 128 DFT Proportion of rows ( δ Ω ) Expected norm Figure 3. S ample av erage of the norm of a random δ Ω n × δ T n sub matrix dra w n from th e 128 × 128 DFT matrix. SPIKES A N D SINES 13 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 Proportion of rows ( δ Ω ) Scaled norm of random rectangular submatrix drawn from 128 × 128 DFT Proportion of columns ( δ T ) Expected norm * max( δ T , δ Ω ) −1/2 Figure 4. S ample av erage of the norm of a random δ Ω n × δ T n sub matrix dra w n from th e 128 × 128 DFT matrix and rescaled by max {| T | , | Ω |} − 1 / 2 . deviation from zero on the in terv al [0 , 1]. The pap er [BK85] is r elev an t in this connection: its auth ors ident ify the non n egat iv e p olynomials with least deviation from zero with resp ect to L p norms for p < ∞ . Th e p = ∞ case app ears to b e op en, and uniqueness ma y b e an issue. (2) Instead of reducing the p r oblem to the square case, it would b e v aluable to un derstand the rectangular case directly . Again, it may b e p ossib le to adapt Prop osition 17 to hand le this situati on. This approac h wo uld probably r equire the biv ariate p olynomials of least deviation fr om zero id en tified by Sloss [Slo65]. (3) A harder p roblem is to determine the limiting b ehavior of th e exp ected norm of a random submatrix as the d imension grows and the p rop ortion of ro ws and columns remains fixed. W e frame the follo win g conjecture. Conjecture 20 (Qu artercircle La w) . A r andom squar e submatrix of the n × n DFT satisfies E   P δ F P ′ δ   ≤ 2 p δ (1 − δ ) . The i ne quality b e c omes an e quality as n → ∞ . One can d ev elop a similar statemen t ab out random rectangular submatrices. At present, ho w ever, th ese conjectures are out of r eac h. (4) Finally , one migh t stu d y the b eha vior of the lo wer sin gu lar v alue of a (suitably n ormalized) random submatrix d ra wn from the DFT. Th ere are some results a v ailable wh en one set, sa y T , is fixed [CR T 06]. It is p ossible that the b eha vior will b e b etter when b oth sets are random. The pr esent methods d o not seem to pro vide m u c h inf ormation ab out this problem. 14 JOEL A. TROPP A cknowledgments One of the anonymous referees pr o vided a w ealth of us eful advice th at substant ially improv ed the qualit y of this w ork. In particular, the referee describ ed a v ersion of L emma 15 and demons tr ated that it offers a simpler route to th e main results than th e argum en t in earlier drafts of this pap er. Appendix A. Chebyshev Extrapola tion One of the ma j or to ols in the pr oof of T heorem 9 is Prop osition 17 . This result extrap olates the momen ts of the norm of a large r andom subm atrix drawn from a fi x ed matrix, giv en information ab out a small random sub matrix. An imp ortan t idea b ehind the result is to fold inform atio n ab out the sp ectral norm of the matrix in to th e estimate . The extrap olatio n tec hnique is due to Bourgain and T zafriri [BT91 ]. W e r equire a v arian t of their result, so w e rep eat the argumen t in its entiret y . The complete statement of the result f ollo ws. Prop osition 21. Supp ose that A is an n × n matrix with k A k ≤ 1 . L et q b e an inte ger that satisfies 13 log n ≤ q ≤ n/ 2 . Cho ose p ar am eters  ∈ (0 , 1) and δ ∈ [ , 1] . F or e ach λ ∈ [0 , 1] , it holds that  E   R δ AR ′ δ   2 q  1 / 2 q ≤ 8 δ λ max  1 ,  − λ  E   R  AR ′    2 q  1 / 2 q  . The same r esult holds if we r eplac e R ′ δ by R δ and r epla c e R ′  by R  . V. A. Mark o v observed that the co efficien ts of an arbitrary p olynomial can b e b ounded in terms of the co efficien ts of a Chebyshev p olynomial b ecause Chebyshev p olynomials are th e unique p olynomials of least d eviatio n fr om zero on the un it inte rv al. S ee [Tim63, Sec. 2.9] for more details. Prop osition 22 (Ma rk o v) . L et p ( t ) = P r k =0 c k t k . The c o efficients of the p olynomial p satisfy the ine quality | c k | ≤ r k k ! max | t |≤ 1 | p ( t ) | ≤ e r max | t |≤ 1 | p ( t ) | . for e ach k = 0 , 1 , . . . , r . With Mark o v’s result at h and, we can pr o ve Prop osition 21. Pr o of of Pr op osition 21. W e establish the resu lt when the tw o diagonal pro jectors are indep en den t; the other case is almost identi cal b ecause this indep endence is nev er exploited. Define the function F ( s ) = E   R s AR ′ s   2 q for s ∈ [0 , 1]. Note that F ( s ) ≤ 1 b ecause k R s AR ′ s k ≤ k A k ≤ 1. F ur th ermore, F d oes not decrease. The f unction F is comparable with a p olynomial. Use the facts th at 2 q is even and that A has dimension n to chec k the inequalities F ( s ) ≤ E trace [( R s AR ′ s ) ∗ ( R s AR ′ s )] q ≤ nF ( s ) . (A.1) Define a second fu n ction p ( s ) = E trace[( R s AR ′ s ) ∗ ( R s AR ′ s )] q = E trace( A ∗ R s AR ′ s ) q , where w e us ed the cyclicit y of th e tr ace and the fact that R s and R ′ s are diagonal matrices with 0–1 en tries. Expand the pro duct and compute th e exp ectation using the additional fact that the en tries of the diagonal matrices are indep endent rand om v ariables of mean s . W e disco ve r that p is a p olynomial of maximum d egree 2 q in the v ariable s : p ( s ) = X 2 q k =1 c k s k The p olynomial has no constan t term b ecause R 0 = 0 . SPIKES A N D SINES 15 W e can use Mark ov’s tec hn ique to b oun d the coefficient s of the p olynomial. First, mak e the c h ange of v ariables s = t 2 to see that    X 2 q k =1 c k  k t 2 k    =   p ( t 2 )   ≤ nF ( t 2 ) ≤ nF (  ) for | t | ≤ 1. The first inequalit y follo ws from (A.1) a nd the second follo ws from the monot onicit y of F . The p olynomial p ( t 2 ) has degree 4 q in the v ariable t , so P rop osition 22 yields | c k |  k ≤ n e 4 q F (  ) for k = 1 , 2 , . . . , 2 q . (A.2) Ev aluate this expression at  = 1 and recall that F ≤ 1 to obtain a s eco nd b oun d, | c k | ≤ n e 4 q for k = 1 , 2 , . . . , 2 q . (A.3) T o complete the pro of, we ev aluat e the p olynomial at a p oin t δ in the range [ , 1]. Fix a v alue of λ in [0 , 1], and set K = ⌊ 2 λq ⌋ . In view of (A.2) and (A.3) , w e obtain F ( δ ) ≤ X K k =1 | c k | δ k + X 2 q k = K +1 | c k | δ k ≤ X K k =1 n e 4 q F (  )( δ/ ) k + X 2 q k = K +1 n e 4 q δ k ≤ n e 4 q  K ( δ / ) K F (  ) + (2 q − K ) δ K +1  ≤ n e 4 q δ 2 λq h K  − 2 λq F (  ) + (2 q − K ) i ≤ n e 4 q δ 2 λq · 2 q max { 1 ,  − 2 λq F (  ) } The thir d and fourth inequalities us e the conditions δ / ≥ 1 and δ ≤ 1, and the last b ound is an application of Jens en ’s inequalit y . T aking the (2 q )th ro ot, we reac h F ( δ ) 1 / 2 q ≤ (2 q n ) 1 / 2 q e 2 δ λ max { 1 ,  − λ F (  ) 1 / 2 q } . The leading constant is less than 8, p ro vided that 13 log n ≤ q ≤ n/ 2.  Referen ces [BK85] V. F. Babenko and V. A. K ofano v. Polynomials of fixed sign that deviate least from zero in the spaces L p . Math. Notes , 37(2):99–105, F eb. 1985. [BT91] J. Bourgain and L. Tzafriri. On a problem of Kadison and S inger. J. r eine angew. Math. , 420:1–43, 1991. [Buc01] A. Buchholz. Operator Khintc h in e inequality in non-commutativ e probability . Math. Annalen , 319:1–16 , 2001. [CR06] E. J. Cand ` es and J. Romberg. Quantitativ e robust un certainty prin cip les and optimally sparse d ecomposi- tions. F oundat ions of Comput. Math , 6:227–254, 2006 . [CR07] E. J. Cand` es and J. Romberg. Sparsity and incoherence in compressive sampling. I nverse Pr oblems , 23:969– 985, June 2007. [CR T06] E. Cand` es, J. Romberg, and T. T ao. Robu st uncertain ty principles: Exact signal reconstruction from h ighly incomplete Fourier information. IEEE T r ans. Info. The ory , 52(2):489–50 9, F eb. 2006. [CT06] E. J. Cand` es and T. T ao. Near-optimal signal recov ery from random pro jections: Univ ersal encod ing strate- gies? IEEE T r ans. I nfo. The ory , 52(12):5406–5425 , D ec. 2006. [Dem97] J. Demmel. Appli e d Numeric al Line ar A lgebr a . SIA M, Philadelphia, 1997. [DL92] D . L. Don oh o and P . Logan. Signal reco very and the large sieve. SIAM J. Appl. Math. , 52(2):577–591, 1992. [DS89] D. L. Donoho and P . B. Stark. Uncertaint y p rin ciples and signal recov ery . SIAM J. Appl. Math. , 49(3):906– 931, June 1989. [EB02] M. Elad and A. M. Bruc kstein. A generalized uncertain ty principle and sparse representation in p airs of bases. IEEE T r ans. I nfo. The ory , 48(9):2558–2567, 2002. [Jam06] G. J. O. Jameson. Notes on the large siev e. Av ailable online: http://www .maths.la ncs.ac.uk/ ~ jameson/ls v.pdf , 2006. [JS68] K. Jogdeo and S. M. S am uels. Monotone converg ence of binomial probabilities and generalization of Ra- manuja n’s equation. Ann. Math. Stat. , 39:1191 –1195, 1968. [LP86] F. Lust-Picqu ard. I n´ egalit ´ es de Kh in tc hine dans C p (1 < p < ∞ ). Comptes R endus A c ad. Sci. Paris, S´ erie I , 303(7):28 9–292, 1986. 16 JOEL A. TROPP [Rau07] H. Rauhut. Random sampling of t rigonometric p olynomials. Appl. Comp. Harmonic Anal. , 22(1):1 6–42, 2007. [Rud99] M. Rud elson. Rand om vectors in the isotropic p osition. J. F unctional Anal. , 164:60– 72, 1999. [R V06] M. Ru delson and R. V eshynin. Sparse reconstru ct ion by conv ex relaxation: Fourier and Gaussian measure- ments. In Pr o c. 40th Annual Confer enc e on Information Scienc es and Systems , Princeton, Mar. 2006. [Slo65] J. M. S loss. Chebyshev appro ximation to zero. Pacific J. Math , 15(1):305–313 , 1965. [T ao 05] T. T ao. An uncertaint y p rin cip le for cyclic groups of prime order. Math. R es. L ett. , 12(1):121 –127, 2005. [Tim63] A. F. Timan. The ory of appr oximation of functions of a r e al variable . Pergamon, 1963. [T ro0 7] J. A. T ropp. O n the conditioning of random subd ictionarie s. Appl. Comp. Harmonic Anal. , 2007. T o app ear. [T ro0 8] J. A. T ropp. The random paving prop ert y for uniformly b ounded matrices. Studia Math. , 18 5(1):67–82 , 2008.