Compressive Shift Retrieval

1 Compressi v e Shift Retrie v al Henrik Ohlsson, Member , IEEE, Y onina C. Eldar , F ellow , IEEE , Allen Y . Y ang, Senior Member , IEEE , and S. Shankar Sastry , F ellow , IEEE Abstract —The classical shift retrie val problem considers two signals in vector form that are related by a shift. The problem is of great importance in many applications and is typically solved by maximizing the cross-corr elation between the two signals. Inspired by compressiv e sensing, in this paper , we seek to estimate the shift directly from compressed signals. W e show that under certain conditions, the shift can be recov ered using fewer samples and less computation compared to the classical setup. Of particular interest is shift estimation from Fourier coefficients. W e show that under rather mild conditions only one Fourier coefficient suffices to recover the true shift. I . I N T RO D U C T I O N S HIFT retriev al is a fundamental problem in many signal processing applications. For example, to map the ocean floor , an acti ve sonar can be used. The sonar transmits certain sound pulse patterns in the water , and the time it takes to receiv e the echoes of the pulses indicates the depth of the ocean floor . In target tracking using two acoustic sensors, the time shift when a sound wa ve of a vehicle reaches the microphones indicates the direction to the vehicle. In the case of a time shift, the shift retriev al problem is often referred to as time delay estimation (TDE) [3]. In computer vision, the spatial shift relating two images is often sought and referred to as image registration or alignment [4], [5], [6]. T raditionally , the shift retriev al problem is solved by max- imizing the cross-correlation between the two signals [7]. In this paper , we revisit this classical problem, and show how the basic premise of compr essive sensing (CS) [8], [9], [10], [11] can be used in the context of shift retriev al. This allows to recover the shift from compressed data leading to computational and storage savings. Compressiv e sensing is a sampling scheme that makes it possible to sample at the information rate instead of the classical Nyquist rate predicted by the bandwidth of the signal [12]. The majority of the results in compressi ve sensing Ohlsson, Y ang and Sastry are with the Dept. of Electrical Engineering and Computer Sciences, University of California, Berkeley , CA, USA, e-mail: ohlsson@eecs.berkeley .edu. Ohlsson is also with Dept. of Electrical Engineering, Link ¨ oping Uni versity , SE-581 83 Link ¨ oping, Sweden. Eldar is with the Dept. of Electrical Engineering, T echnion – Israel Institute of T echnology , Haifa 32000, Israel. Ohlsson is partially supported by the Swedish Research Council in the Linnaeus center CADICS, the European Research Council under the ad- vanced grant LEARN, contract 267381, by a postdoctoral grant from the Sweden-America Foundation, donated by ASEA ’s Fellowship Fund, and by a postdoctoral grant from the Swedish Research Council. Eldar is supported in part by the Israel Science Foundation under Grant no. 170/10, and by the Ollendorf Foundation. Y ang is supported in part by AR O 63092-MA-II, D ARP A F A8650-11-1-7153 and ONR N00014-13-1-0341. This paper was presented in part at the 38th International Conference on Acoustics, Speech, and Signal Processing (ICASSP), May 26-31, 2013, [1]. See also [2]. discuss conditions and methods for guaranteed reconstruction from an under-sampled version of the signal. Therefore, the information rate is typically referred to as the one that guar- antees the recovery of the sparse signal. Howe ver , for many applications such as the aforementioned examples in shift retrie val, obtaining the signal may not be needed. The goal is to recover some properties or statistics of the unkno wn signal. T aking the activ e sonar for example, one may wonder if it is really necessary to sample at a rate which is twice that of the bandwidth of the transmitted signal so that the recei ved signal can be exactly reconstructed? Clearly the answer is no. Since the signal itself is not of interest to the application, we might consider an alternativ e sampling scheme to directly estimate the shift without first reconstructing the signal. These ideas have in fact been recently explored in the context of radar and ultrasound [13], [14], [15], [16] with continuous time signals and multiple shifts. Here we consider a related problem and ask: What is the minimal information rate to shift r etrieval when two related discrete-time signals ar e under-sampled? It turns out that under rather mild conditions, we only need fractions of the signals. In fact, we will show that only one Fourier coef ficient from each of the signals suffices to recov er the true shift. W e refer to the method as compr essive shift r etrieval (CSR). It should be made clear that CSR does not assume that any of the in volv ed signals are necessarily sparse. As the main contribution of the paper, we will sho w that when the sensing matrix is taken to be a partial Fourier matrix, under suitable conditions, the true shift can be recovered from both noise-free and noisy measurements using CSR. Furthermore, CSR reduces both t he computational load and the number of samples needed in the process. This is of particular interest since recent dev elopments in sampling [17], [18], [19] hav e shown that Fourier coef ficients can be efficiently obtained from space (or time) measurements by the use of an appropriate filter and by subsampling the output. Remarkably , our results also show that in some cases sampling as few as one Fourier coefficient is enough to perfectly recov er the true shift. A. Prior W ork Compressiv e signal alignment problems ha ve been ad- dressed in only a few publications and, to the authors’ best knowledge, not in the same setup studied in this paper . In [20], the authors considered alignment of images under random projection. The work was based on the Johnson-Lindenstrauss property of random projection and proposed an objective function that can be solv ed efficiently using dif ference-of- two-con vex pr ogramming algorithms. In this paper , we instead 2 focus on proving theoretical guarantees of exact shift recovery when the signal is subsampled by a partial Fourier basis. The theory developed in [20] does not apply to this setup. The smashed filter [21] is another related technique. It is a general frame work for maximum likelihood hypothesis testing and can be seen as a dimensionally reduced matched filter . It can therefore be applied to the shift retriev al problem. The underlying idea of both the smashed filter and CSR are the same in that both approaches try to av oid reconstructing the signal and extract the sought descriptor , namely , the shift, from compressiv e measurements. Ho wev er, the analysis and assumptions are very dif ferent. For CSR, we dev elop requirements for guaranteed recov ery of the true shift for a giv en measurement matrix. For the smashed filter , the analysis focuses on random orthoprojections and provides probabilities for correct recovery as a function of the number of projections. Also, in this paper , we are particularly interested in F ourier measurements, and many of the results are therefore tailored to this setting. The work presented here can therefore be seen as complementary to what was presented in [21] and its extension in [22]. B. Notation and Assumptions W e use normal fonts to represent scalars and bold fonts for vectors and matrices. The notation | · | represents the absolute value for scalars, vectors and matrices, and it returns the cardinality of a set if the ar gument is a set. F or both vectors and matrices, k · k 0 is the ` 0 -norm function that returns the number of nonzero elements of its argument. Similarly , k · k p represents the ` p -norm. For a vector x , the ` p -norm is defined as k x k p , ( P i | x i | p ) 1 /p , where x i is the i th element of x . For a matrix X , k · k p is defined as X , ( P i,j | X i,j | p ) 1 /p , where X i,j is the ( i, j ) -th element of X . Furthermore, X ∗ denotes the complex conjugate transpose of X . Let I n × n denote an n × n identity matrix, 0 m × n an m × n matrix of zeros, and Z be the set of integers. <{·} returns the real part of its argument. W e say that tw o n -dimensional vectors y and x are related by an l cyclic-shift if y = D l x , where D l is defined as D l =  0 l × ( n − l ) I l × l I ( n − l ) × ( n − l ) 0 ( n − l ) × l  . (1) Throughout the paper , we will assume that the shift is unique up to a multiple of n . Also note that we are considering cyclic shifts. C. Organization In Sections II and III, we study the CSR problem under the assumption that the measurements are noise free. Next, we extend the results to noisy measurements in Section IV. As we are particularly interested in Fourier measurements, we will tailor the results to this particular choice of sensing matrix. Section V concludes the paper . All proofs are provided in the Appendice for clarity . I I . N O I S E - F R E E C O M P R E S S I V E S H I F T R E T R I E V A L The shift retriev al problem is a multi-hypothesis testing problem: Define the s th hypothesis H s , s = 0 , . . . , n − 1 , as H s : x is related to y via a s -cyclic-shift , and accept H s if y = D s x and otherwise reject. Since the true shift is assumed unique, only one hypothesis will be accepted and the corresponding shift is necessarily the true solution. The connection to the cross-correlation is now trivial: k y − D s x k 2 2 = k y k 2 2 + k D s x k 2 2 − y ∗ D s x − D s x ∗ y = k y k 2 2 + k x k 2 2 − 2 <{h y , D s x i} (2) where we use the fact that k D s x k 2 2 = k x k 2 2 . Since k y − D s x k 2 2 ≥ 0 , equating y = D s x is equiv alent to minimizing k y − D s x k 2 2 or maximizing the real part of the cross- correlation with respect to s : max <{h y , D s x i} . (3) Now , assume that the compressed measurement signals z = Ay ∈ C m and v = Ax ∈ C m are giv en and related to the ground-truth signals x ∈ C n and its shifted version y = D l x ∈ C n via the sensing matrix A ∈ C m × n , m < n . The goal of CSR is to recov er the shift l relating x and y from the compressed measurements z and v . Since only the compressed measurements z and v are assumed av ailable, we can not e v aluate y = D s x or maximize <{h y , D s x i} for each hypothesis D s . Ho wev er, if A ∗ A and D s commute for all s = 0 , . . . , n − 1 , then y = D s x ⇒ A ∗ Ay = A ∗ AD s x = D s A ∗ Ax ⇔ A ∗ z = D s A ∗ v . (4) Hence, we could consider the test: Accept H s if A ∗ z = D s A ∗ v and otherwise reject . (5) It is clear that if s is such that y = D s x , then A ∗ z = D s A ∗ v will also hold. Howe ver , the other way around might not be true. Therefore, we might erroneously accept a wrong hypotheses using (5). The next theorem lists the conditions by which the testing (5) is guaranteed to accept the correct hypothesis. Notice that testing the condition A ∗ z = D s A ∗ v is equiv alent to minimizing k A ∗ z − D s A ∗ v k 2 2 with respect to s . Theorem 1 ( Shift Recovery from Low-Rate Data). Let X be an n × n matrix with the i th column equal to D i x , i = 1 , . . . , n, and define ¯ D s = AD s A ∗ . If the sensing matrix A satisfies the following conditions: 1) A ∗ AD s = D s A ∗ A , 2) ∃ α ∈ R , α AA ∗ = I and 3) all columns of AX are differ ent, then max s <{h z , ¯ D s v i} (6) or equivalently the test (5) r ecovers the true shift. The conditions of Theorem 1 may seem restrictive. How- ev er, as we will show in Lemma 3, if A is chosen as a partial Fourier matrix, then the first two conditions of Theorem 1 are trivially satisfied. The last condition is the only one that needs 3 to be checked and will lead to a condition on the sampled Fourier coefficients. The conditions of Theorem 1 can be checked prior to estimating the shift. Howe ver , kno wing the estimate of the shift, it is easy to see from the proof (see the proof of Lemma 8) that it is enough to check if the column of AX associated with the estimate of the shift is different than all the other columns of AX . Hence, we do not need to check if all columns of AX are different. This conclusion is formulated in the follo wing corollary , which is less conserv ati ve than Theorem 1. Corollary 2 ( T est for T rue Shift). Let X be an n × n matrix with the i th column equal to D i x , i = 1 , . . . , n, and define ¯ D s = AD s A ∗ . If the sensing matrix A satisfies the following conditions: 1) A ∗ AD s = D s A ∗ A , and 2) ∃ α ∈ R , α AA ∗ = I , then s ∗ = arg max s <{h z , ¯ D s v i} (7) is the true shift if the s ∗ th column of AX is differ ent than all the other columns of AX . I I I . C O M P R E S S I V E S H I F T R E T R I E V A L U S I N G F O U R I E R C O E FFI C I E N T S Of particular interest is the case where A is made up of a partial Fourier basis. That is, A takes the form A = 1 √ n        1 e − 2 j πk 1 n e − 4 j πk 1 n · · · e − 2( n − 1) j πk 1 n 1 e − 2 j πk 2 n . . . e − 2( n − 1) j πk 2 n . . . . . . 1 e − 2 j πk m n e − 4 j πk m n · · · e − 2( n − 1) j πk m n        where k 1 , . . . , k m ∈ { 0 , 1 , 2 , . . . n − 1 } , m ≤ n . For this specific choice, AX = 1 √ n        X k 1 X k 1 e 2 k 1 πj n · · · X k 1 e 2( n − 1) k 1 πj n X k 2 . . . X k 2 e 2( n − 1) k 2 πj n . . . X k m X k m e 2 k m πj n · · · X k m e 2( n − 1) k m πj n        where X r denotes the r th F ourier coefficient of the F ourier transform of x . For a sensing matrix made up by a partial Fourier basis, we hav e the following useful result: Lemma 3. Let A be a partial F ourier matrix. Then D s A ∗ A = A ∗ AD s for all s = 1 , . . . , n . Using this result in Theorem 1 giv es the following corollary: Corollary 4 ( Shift Recovery fr om Low Rate Fourier Data). W ith A denoting a partial F ourier matrix and z i and v i the i th element of z and v , max s < ( m X i =1 z i v i e − 2 πj k i s n ) (8) r ecovers the true shift if there exists p ∈ { 1 , . . . , m } such that X k p 6 = 0 and { 1 , . . . , n − 1 } k p n contains no integ ers. In particular , measuring only the first F ourier coefficients ( k 1 = 1 ) of x and y would, as long as the coefficients ar e nonzer o, suffice to r ecover the true shift. Remarkably , in the extreme case when m = 1 , the corollary states that all we need is two scalar measurements, z and v , to perfectly recov er the true shift. The scalar measurements can be any nonzero Fourier coefficient of x and y as long as { 1 , . . . , n − 1 } k 1 n contains no integers. As noted in the corollary , the first Fourier coefficients ( k 1 = 1 ) of x and y would suffice. Also note that only 2 mn multiplications are required to e valuate the test. This should be compared with n 2 multiplications to ev aluate the cross-correlation for the full uncompressed signals x and y . Corollary 4 is easy to check but more conservati ve than both Theorem 1 and Corollary 2. T o validate the results, we carried out the following ex- ample. In each trial we let the sample dimension m and the shift l be random integers between 1 and 9 and generate x by sampling from a n -dimensional uniform distribution. W e let n = 10 and make sure that A in each trial is a partial Fourier basis satisfying the assumptions of Corollary 4. W e carry out 10000 trials. The true shift is successfully recovered in each trial by the simplified test (8), namely , with 100% success rate. This is quite remarkable since when m = 1 , we recover the true shift using only two scalar measurement z and v and 1 / 5 of the multiplications that maximizing the real part of inner product between the original signals (3) would need. I V . N O I S Y C O M P R E S S I V E S H I F T R E T R I E V A L Now we consider the noisy version of compressive shift retriev al, where the measurements z and v are perturbed by noise: ˜ z = z + e z , ˜ v = v + e v . (9) Similar to the noise-free case, here we can also guarantee the recov ery of the true shift. Our main result is giv en in the following theorem: Theorem 5 ( Noisy Shift Recov ery fr om Low-Rate Data). Let ˜ x be such that ˜ v = A ˜ x , the i th column of ˜ X be shifted versions of ˜ x , and assume that A is a partial F ourier matrix and that the noisy measur ements ar e used in (8) to estimate the shift. If the ` 2 -norm differ ence between any two columns of A ˜ X is gr eater than ∆ zv , k e z k 2 + k e v k 2 + q k ˜ v k 2 2 + k ˜ z k 2 2 − 2 max s <{h ˜ z , ¯ D s ˜ v i} , then the estimate of the shift is not affected by the noise. Note that the theorem only states that the noise does not affect the estimate of the shift. It does not state that the shift will be the true shift. W e illustrate the results by running a Monte Carlo simu- lation consisting of 10000 trials for each sample dimension m = 1 , . . . , 10 , and for two different SNR levels. In Figure 1, 10 histograms are shown (corresponding to m = 1 , . . . , 10 ) for the SNR = k z k 2 2 k ˜ z − z k 2 2 (10) 4 being 2 (low SNR) and in Figure 2, S N R = 10 (high SNR). The errors e z and e x were both generated by sampling from N (0 , σ 2 ) + j N (0 , σ 2 ) . (11) W e further use n = 10 , l = 5 and sample x from a uniform (0,1)-distribution. The conclusion from the simulation is that the smaller the m , the more the estimate of the shift is sensiti ve to noise. Notice that when m = 10 the test (8) reduces to the classical test of maximizing the cross-correlation. W e can now use Theorem 5 to check if the noise affected the estimate of the shift or not in each of the trials. F or m = 2 and the high SNR, 40% of the trials satisfied the conditions of Theorem 5 and the noise therefore did not af fect the shift estimates in those cases. Of the trails that satisfied the conditions, all predicted the true shift and none a false shift. Note ho wever that Theorem 5 only states that if the conditions are satisfied, then the estimated shift is the same as if we would hav e used the noise free compressed measurements in the test (8). It does not state that the estimate will be the true shift. Fig. 1. Histogram plots for the estimated shift and low SNR. From left to right, top to bottom, m = 1 , . . . , 10 . The true shift was set to 5 in all trials. Fig. 2. Histogram plots for the estimated shift and high SNR. From left to right, top to bottom, m = 1 , . . . , 10 . The true shift was set to 5 in all trials. Theorem 5 gi ves conditions for when the noise does not affect the estimate of the shift. This is a good property but ev en better would be if the recovery of the true shift could be guaranteed. This is given by the follo wing corollary . Corollary 6 ( Recovery of the T rue Shift from Noisy Low-Rate Data). If the ` 2 -norm differ ence between any two columns of A ˜ X is gr eater than 2 k e v k 2 and the conditions of Theor em 5 are fulfilled, then (8) recover s the true shift. If the estimate of the shift has been computed, a less conservati ve test can be used to check if the computed estimate has been af fected by noise and if it is the true one. W e summarize our conclusion in the following corollary . Corollary 7 ( T est for T rue Shift in the Presence of Noise). Assume that (8) gives s ∗ as an estimate of the shift. If the ` 2 differ ence between any column and the s ∗ -column of A ˜ X is gr eater than 2 k e v k 2 and ∆ z v , then s ∗ is the true shift. V . C O N C L U S I O N T o reco ver the cyclic shift relating two 1D signals, the cross-correlation is usually ev aluated for all possible shifts. Recent advances in hardware, signal acquisition and signal processing hav e made it possible to sample or compute Fourier coefficients of a signal efficiently . It is therefore of particular interest to see under what conditions the true shift can be recov ered from the Fourier coefficients. W e hav e proposed a criterion that is computationally more efficient than using the time samples, and we have shown that the true shift can be recov ered using as few as one Fourier coefficient. W e hav e also deriv ed bounds for perfect recovery for both noise free and noisy measurements. A P P E N D I X A P R O O F S : N O I S E - F R E E C O M P R E S S I V E S H I F T R E T R I E V A L Before proving the Theorem 1, we state two lemmas. Lemma 8 ( Recovery of Shift using Projections). Let X be the n × n -matrix made up of cyclically shifted versions of x as columns. If the columns of AX are distinct, then the true shift can be reco ver ed by min q ∈{ 0 , 1 } n k Ay − AX q k 2 2 s.t. k q k 0 = 1 . (12) Proof of Lemma 8: Since the shift relating x and y is assumed unique, it is clear that the true shift is recov ered by min q ∈{ 0 , 1 } n k y − X q k 2 2 s.t. k q k 0 = 1 . (13) Assume that the solution of (12) is not equiv alent to that of (13). Namely , assume that (13) giv es q , (12) giv es ˜ q and q 6 = ˜ q . Since q will give a zero objecti ve value in (12), so must ˜ q . W e therefore hav e that Ay = AX ˜ q = AX q and hence AX ˜ q − AX q = AX ( ˜ q − q ) = 0 . (14) Since q , ˜ q ∈ { 0 , 1 } n , k ˜ q k 0 = k q k 0 = 1 , and q 6 = ˜ q , AX ( ˜ q − q ) = 0 implies that two columns of AX are identical. This is a contradiction and we therefore conclude that both (12) and (13) recov er the true shift. Lemma 9 ( From (12) to (6) ). Under conditions 1) and 2) of Theor em 1, the shifts recover ed by (12) and (6) ar e the same. Proof of Lemma 9: Consider the objective of (12): k Ay − AX q k 2 2 =( Ay ) ∗ Ay + ( AX q ) ∗ AX q − ( Ay ) ∗ AX q − ( AX q ) ∗ Ay . (15) Writing X q = D s x , problem (12) is equal to max s 2 <{ ( Ay ) ∗ AD s x } − ( AD s x ) ∗ AD s x . (16) Now , if A ∗ AD s = D s A ∗ A and using that ( D s ) ∗ D s = I for a shift matrix, then ( AD s x ) ∗ AD s x = x ∗ ( D s ) ∗ A ∗ AD s x = k Ax k 2 2 , (17) which is independent of s . Therefore, the shift recovered by (16) is the same as that of max s <{ ( Ay ) ∗ AD s x } . (18) 5 Lastly , if we ag ain use that A ∗ AD s = D s A ∗ A and α AA ∗ = I , then (6) follows from <{ ( Ay ) ∗ AD s x } = <{ y ∗ A ∗ AD s x } (19) = α <{ y ∗ A ∗ AA ∗ AD s x } (20) = α <{ y ∗ A ∗ AD s A ∗ Ax } (21) = α <{h z , ¯ D s v i} (22) where z = Ay and v = Ax . W e are now ready to prove Theorem 1. Proof of Theorem 1: The assumptions of Theorem 1 imply that requirements of both Lemmas 8 and 9 are satisfied. The theorem therefore follows trivially . W e next prove Corollary 2. Proof of Corollary 2: In the proof of Lemma 8, AX ( ˜ q − q ) = 0 leads to ˜ q − q = 0 if the columns of AX were all distinct. Now , if s ∗ = arg max s <{h z , ¯ D s v i} , (23) that corresponds to the s ∗ th element of ˜ q being one and all other elements zero. Hence, Lemma 8 can be made less conservati ve if s ∗ is kno wn by requiring that only the s ∗ th column of AX is different than all other columns. Proof of Lemma 3: Let M = AD s and Q = A ( D s ) ∗ . By the definition of D s , M is a column permutation of A where the columns are shifted s times to the right. Thus, the r th column of M is equal to the t th column of A where t = ( r − s ) mod n . It is also easy to see that ( D s ) ∗ permutes the columns of A by s to the left so that the r th column of Q is equal to the q th column of A where q = ( r + s ) mod n . Now , the pr th element of A ∗ M = A ∗ AD s is gi ven by ( A : ,p ) ∗ M : ,r = ( A : ,p ) ∗ A : ,r − s = 1 n m X i =1 e 2 j π k i ( p − r + s ) , (24) where A : ,p is used to denote the p th column of A and M : ,r the r th column or M . On the other hand, the ( p, r ) -th element of Q ∗ A = D s A ∗ A is given by ( Q : ,p ) ∗ A : ,r = ( A : ,p + s ) ∗ A : ,r = 1 n m X i =1 e 2 j π k i ( p + s − r ) . (25) Clearly , the two are equi valent. W e are now ready to prove Corollary 4. Proof of Corollary 4: Lemma 3 giv es that Condition 1) of Theorem 1 is satisfied. Since a full Fourier matrix is orthonormal, a matrix made up of a selection of rows of a Fourier matrix satisfies Condition 2). The last condition of Theorem 1 requires columns of AX to be distinct. A sufficient condition is that there exists a row with all distinct elements. As shown pre viously , the ( p, r ) -th element of AX is X k p e 2 j πk p ( r − 1) n . If X k p is assumed nonzero, a sufficient condition for AX to have distinct columns is that e 2 j πk p r 1 n 6 = e 2 j πk p r 2 n , r 1 , r 2 ∈ { 0 , . . . , n − 1 } , r 1 6 = r 2 . This condition can be simplified to k p r 1 n 6 = k p r 2 n + γ , γ ∈ Z . By realizing that r 1 − r 2 takes values in {− n + 1 , . . . , − 1 , 1 , . . . , n − 1 } we get that the condition is equiv alent to requiring that there is no integers in {− n + 1 , . . . , − 1 , 1 , . . . , n − 1 } k p n . Due to symmetry , a suf ficient condition for distinct columns is that there exists a p ∈ { 1 , . . . , m } such that X k p 6 = 0 and { 1 , . . . , n − 1 } k p n contains no integers. Lastly , if we write out AD s A ∗ we get that the pr th element is equal to δ p,r e − 2 j πk p s n /n and hence the simplified test proposed in (8). A P P E N D I X B P R O O F S : N O I S Y C O M P R E S S I V E S H I F T R E T R I E V A L Proof of Theorem 5: From Lemma 9 we can see that seeking s that maximizes <{h ˜ z , ¯ D s ˜ v i} is equiv alent to seeking q that solves min q ∈{ 0 , 1 } n k ˜ z − A ˜ X q k 2 2 s.t. k q k 0 = 1 , (26) where the first column of A ˜ X is equal to ˜ v (which defines the first column of ˜ X ) and the i th column of ˜ X a circular shift of the first column i − 1 steps. Assume that ˆ q solves (26). Since our measurements are noisy , we can not expect a zero loss. The loss can be shown giv en by k ˜ z − A ˜ X ˆ q k 2 2 = k ˜ v k 2 2 + k ˜ z k 2 2 − max s 2 <{ ˜ z ∗ ¯ D s ˜ v } . (27) Now , consider k ˜ z − A ˜ X ˆ q k 2 . Assume that q 0 solves the noise- free version of (26) and let ˜ X = X + H . W e have the following inequality: k ˜ z − A ˜ X ˆ q k 2 = k z + e z − z + AX q 0 − A ˜ X ˆ q k 2 = k e z + AX q 0 − A ˜ X ˆ q k 2 = k e z + A ( ˜ X − H ) q 0 − A ˜ X ˆ q k 2 ≥ k A ˜ X q 0 − A ˜ X ˆ q k 2 − k e z k 2 − k e v k 2 , where we used the fact that AH q 0 = e v . Therefore k A ˜ X q 0 − A ˜ X ˆ q k 2 ≤ ∆ z v . (28) Since k ˆ q k 0 = k q 0 k 0 = 1 , if the ` 2 difference between an y two columns of A ˜ X is greater than ∆ v z , then q 0 = ˆ q . Proof of Corollary 6: Let ˜ q and ˆ q be any vectors such that k ˆ q k 0 = k ˜ q k 0 = 1 , ˆ q 6 = ˜ q and ˆ q , ˜ q ∈ { 0 , 1 } n . Using the triangle inequality we have that k A ˜ X ˆ q − A ˜ X ˜ q k 2 = k A ( X + H )( ˆ q − ˜ q ) k 2 (29) ≤k AX ( ˆ q − ˜ q ) k 2 + k AH ( ˆ q − ˜ q ) k 2 (30) ≤k AX ( ˆ q − ˜ q ) k 2 + 2 k e v k 2 . (31) Hence, if k A ˜ X ˆ q − A ˜ X ˜ q k 2 − 2 k e v k 2 > 0 , then k AX ( ˆ q − ˜ q ) k 2 is greater than zero. No w since Theorem 5 gi ves that (8) recov ers the same shift as if the measurements would ha ve been noise-free, and since Theorem 1 giv es that the noise-free estimate is equal to the true shift if k AX ( ˆ q − ˜ q ) k 2 is greater than zero (or equiv alent that all columns of AX are distinct), we can guarantee the recovery of the true shift also in the noisy case. Proof of Corollary 7: The corollary follows tri vially by setting the s ∗ th element of ˆ q to one and all other elements zero in the proofs of Theorem 5 and Corollary 6. 6 R E F E R E N C E S [1] H. Ohlsson, Y . C. Eldar, A. Y . Y ang, and S. S. Sastry , “Compressive shift retriev al, ” in the 38th International Conference on Acoustics, Speech, and Signal Processing (ICASSP) , V ancouver, Canada, May 2013. [2] ——, “Compressiv e shift retriev al, ” CoRR , vol. abs/1303.4996, 2013. [3] G. Carter and C. Knapp, “Time delay estimation, ” in Pr oceedings of the IEEE International Conference on Acoustics, Speech, and Signal Pr ocessing , 1976. [4] B. Lucas and T . Kanade, “ An iterativ e image registration technique with an application in stereo vision, ” in Int’l J. Conf. Artificial Intelligence , 1981. [5] G. Hager and P . Belhumeur , “Efficient region tracking with parametric models of geometry and illumination, ” IEEE Tr ansactions on P attern Analysis and Machine Intelligence , vol. 10, pp. 1025–1039, 1998. [6] Y . Peng, A. Ganesh, J. Wright, and Y . Ma, “RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images, ” in Pr oceedings of the IEEE International Conference on Computer V ision and P attern Recognition , 2010. [7] A. Hero, H. Messer, J. Goldberg, D. Thomson, M. Amin, G. Giannakis, A. Swami, J. T ugnait, A. Nehorai, A. Swindlehurst, J.-F . Cardoso, L. T ong, and J. Krolik, “Highlights of statistical signal and array processing, ” IEEE Signal Pr ocessing Magazine , v ol. 15, no. 5, pp. 21– 64, 1998. [8] E. J. Cand ` es and T . T ao, “Decoding by linear programming, ” IEEE T ransactions on Information Theory , v ol. 51, no. 12, 2005. [9] E. J. Cand ` es, J. Romberg, and T . T ao, “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency informa- tion, ” IEEE T ransactions on Information Theory , vol. 52, pp. 489–509, Feb . 2006. [10] D. Donoho, “Compressed sensing, ” IEEE T ransactions on Information Theory , v ol. 52, no. 4, pp. 1289–1306, Apr . 2006. [11] Y . C. Eldar and G. Kutyniok, Compressed Sensing: Theory and Appli- cations . Cambridge Uni versity Press, 2012. [12] M. Mishali and Y . C. Eldar , “Sub-Nyquist sampling, ” IEEE Signal Pr ocessing Magazine , v ol. 28, no. 6, pp. 98–124, 2011. [13] T . Chernyakov a and Y . C. Eldar, “Fourier domain beamforming: The path to compressed ultrasound imaging, ” IEEE T ransactions on Ultra- sonics, F erroelectrics, and F r equency Contr ol , Jul. 2013, submitted. [14] O. Bar-Ilan and Y . C. Eldar, “Sub-Nyquist radar via doppler focusing, ” IEEE T ransactions on Signal Pr ocessing , Nov . 2012, submitted. [15] E. Baransky , G. Itzhak, I. Shmuel, N. W agner, E. Shoshan, and Y . C. Eldar , “ A sub-Nyquist radar prototype: Hardware and algorithms, ” IEEE T ransactions on Aer ospace and Electr onic Systems, special issue on Compr essed Sensing for Radar , Aug. 2012, submitted. [16] M. V etterli, P . Marziliano, and T . Blu, “Sampling signals with finite rate of innovation, ” IEEE Tr ansactions on Signal Pr ocessing , vol. 50, no. 6, pp. 1417–1428, Jun. 2002. [17] R. Tur , Y . C. Eldar, and Z. Friedman, “Innovation rate sampling of pulse streams with application to ultrasound imaging, ” IEEE T ransactions on Signal Pr ocessing , vol. 59, no. 4, pp. 1827–1842, Apr . 2011. [18] K. Gedalyahu, R. T ur, and Y . C. Eldar, “Multichannel sampling of pulse streams at the rate of innovation, ” IEEE T ransactions on Signal Pr ocessing , vol. 59, no. 4, pp. 1491–1504, Apr . 2011. [19] E. Baransky , G. Itzhak, I. Shmuel, N. W agner, E. Shoshan, and Y . C. Eldar , “ A sub-Nyquist radar prototype: Hardware and algorithms, ” IEEE T ransactions on Aer ospace and Electr onic Systems, special issue on Compr essed Sensing for Radar , Aug. 2012, submitted to. [20] E. Kokiopoulou, D. Kressner, and P . Frossard, “Optimal image alignment with random measurements, ” in EUSIPCO , 2009. [21] M. A. Davenport, M. F . Duarte, M. W akin, J. N. Laska, D. T akhar, K. Kelly , and R. G. Baraniuk, “The smashed filter for compressiv e classification and target recognition, ” in Proc. Computational Imaging V at SPIE Electr onic Imaging , 2007. [22] M. A. Davenport, P . T . Boufounos, M. B. W akin, and R. G. Baraniuk, “Signal processing with compressive measurements, ” IEEE Journal of Selected T opics in Signal Pr ocessing , v ol. 4, no. 2, pp. 445–460, 2010.