Improved Deterministic Length Reduction

Impro v ed Deterministic Length Reduction Amiho o d Amir ∗ Klim Efremenk o † Oren Kapah ‡ Bar-Ila n Univ ers it y Bar-Ila n Univ ers it y Bar-Ila n Univ ers it y and Johns Hopkins Univ ersity Ely P orat § Amir Rothsc hild ¶ Bar-Ila n Univ ers it y Bar-Ila n Univ ers it y Abstract This pap er presen ts a new tec hnique for deterministic length reduction. This techn ique improv es the running time of the alg orithm presented in [ ? ] for performing fast convolution in sparse da ta. While the r egular fa st conv olution of v ectors V 1 , V 2 whose size s ar e N 1 , N 2 resp ectively , tak es O ( N 1 log N 2 ) using FFT, using the new technique for length reduction, t he algorithm prop osed in [ ? ] per forms the con volution in O ( n 1 log 3 n 1 ), where n 1 is the n um b er of non-zero v alues in V 1 . The algorithm assumes that V 1 is giv en in adv ance, and V 2 is given in running time. The novel tec hnique presen ted in this pap er improv es the con volution time to O ( n 1 log 2 n 1 ) deterministically , which equals the b est running time given achiev ed by a randomized algor ithm. The pr epro cessing time o f the new technique remains th e same as the prepro cessing time of [ ? ], which is O ( n 2 1 ). This assumes and deals the case wher e N 1 is p olynomial in n 1 . In the case where N 1 is exponential in n 1 , a reduction to a p olynomial case can b e used. In this paper we als o improve the prepro cess ing time o f t his re ductio n from O ( n 4 1 ) to O ( n 3 1 po lylog( n 1 )). 1 In tro duction The d-Dimensional p oint set matching pr oblem serves as p ow erful to ols in numerous application domains. In th e d-Dimensional point set mat c hing problem, t w o sets of points T , P ∈ N d consisting of n, m p oin ts, r esp ectiv ely , are giv en. The goal is to d etermine if there is a r igid transformation ∗ Department of Computer Science, Bar-Ilan Universit y , Ramat-Gan 52900 , Israel, +972 3 531-8770 ; amir@cs.bi u.ac.il ; and Departmen t of Computer Science, Jo hns H opkins Universit y , Baltimore, MD 21218. Partly supp orted by ISF gran t 35 /05. † Department of C omputer Scie nce, Bar-Ilan U., 52900 R amat-Gan, I srael, (972-3)531-8408, klimefrem @gmail.com . ‡ Department of Computer Science, Bar-Ilan U., 52900 Ramat-Gan, Israel, (972-3)531-8408, kapaho@cs.biu.ac .il . § Department of Computer S cience, Bar-Ilan Universit y , 52900 Ramat-Gan, I srael, (972-3)531-76 20; porately@c s.biu.ac.il . ¶ Department of Computer S cience, Bar-Ilan Universit y , 52900 Ramat-Gan, I srael, (972-3)531-76 20; amirrot@gm ail.com . 1 under whic h all the points in P are co v ered with p oin ts in T . Among the important application domains to which this problem con tributes are mo del based ob ject r ecognition, image registration, pharmacophore iden tification, and searc hin g in m usic a rchiv es. F or an exp lanation of the uses o f the p oint- set matc hing problem in these d omains see [ ? ]. The p oint -set matc hing problem has b een studied in the literature in many v ariation, not the least of which in the algorithms literature. In [ ? ] Card oze and Sch ulman used a randomized algorithm to redu ce the sp ace size of T , P and then apply solv e the problem in the r educed space. In [1] Cole and Hariharan prop osed a solution to the d-Dimensional Sp arse Wildc ar d M atching . This is a generalization of th e d -Dimensional p oint set matc hin g p r oblem wher e ev ery p oin t in N d is asso ciated with a v alue. A matc h is declared if the v alues of coinciding p oin ts are equal. The Cole and Hariharan solution c onsists of tw o steps. The first step is a Dimension R e duction w here the in puts T , P are linearized in to raw vect ors T ′ , P ′ of size p olynomial in the num b er of n on-zero v alues. T he second step wa s a L ength R e duction where eac h of the r a w vecto rs T ′ , P ′ w as rep laced b y log n sh ort v ectors of size O ( n ) where n is the num b er of n on-zeros. Th e idea is th at the mappin g to the short v ectors preserves the distances in the original v ectors, thus th e problem is reduced t o a matc h ing pr oblem of short v ectors, to wh ic h efficien t solutions exist. The problem with the length r e duction idea is that more then one p oin t can b e mapp ed in to the same location, th us it is no longer clear whether there is in d eed a matc h in th e original v ectors. Th e prop osed sol ution of Cole and Hariharan w as to create a set of log n pairs of ve ctors usin g log n hash fun ction r ather then a single pair of vec tors. Th eir scheme redu ced the failure pr obabilit y . In [ ? ], the fi rst deterministic algorithm f or finding log n hash fu nctions that redu ce the size of the v ectors to O ( n log n ) wa s presented. The algorithm gu aranteed that eac h non-zero v alue app ears with no colli sions in at le ast on e of the vec tors, thus eliminati ng the p ossibilit y of en error. The length r e duction idea wa s used to solv e the Sp arse Convolution problem p osed in [3], where the aim is to find the con v olution ve ctor W of tw o vec tors V 1 , V 2 whose sizes are N 1 , N 2 , with n 1 , n 2 non-zero elemen ts r esp ectiv ely (wh ere n 1 > n 2 ). It is assu med that the tw o vect ors are not giv en e xplicitly , rather they are giv en as a set of ( index, v alue ) pairs. Using the F ast F ourier T ransform (FFT) a lgorithm, the con v olution can b e cal culated in runn in g time O ( N 1 log N 2 )[2]. In our cont ext, though, the vect ors V 1 , V 2 are spars e. The a im of the algorithm is to compute W in time prop ortional to the num b er o f non -zero entries in W , whic h ma y b e significan tly smaller than O ( N 1 ). Clearly , th is can b e easily d one in time O ( n 1 n 2 ). The goal of the length redu ction is as follo ws: Given t w o v ectors V 1 , V 2 whose s izes are N 1 , N 2 , with n 1 , n 2 non-zero ele ments resp ectiv ely (where n 1 > n 2 ), obtain t wo v ectors V ′ 1 , V ′ 2 of size O ( n 1 ) suc h that all the non-zero in V 1 and in v 2 will app ear as singletons in V ′ 1 and in V ′ 2 resp ectiv ely while main taining th e distance prop erty . The distance pr op ert y w hic h need to b e maintained is defined as follo ws: If V ′ 2 [ f (0)] is ali gned with V ′ 1 [ f ( i )], then V ′ 2 [ f ( j )] will b e aligned w ith V ′ 1 [ f ( i + j )]. This goal w as not reac hed y et, rather a set of O (log n 1 ) v ectors of size O ( n 1 log n 1 ) where obtained in [ ? ], where eac h non-zero in the text app ears at least once as a singleton in the set of v ectors. This length r eduction ga v e an O ( n 1 log 3 n 1 ) alg orithm for con v olution in sparse data. In this pap er w e go one step forwa rd and reduce t he size of the obtained vect ors to O ( n 1 ). This length reduction tec h nique impro ve s the ru nning time of the fast con v olution presen ted in [ ? ] to O ( n 1 log 2 n 1 ), which is th e runnin g time for the randomized algorithm presen ted in [1]. 2 2 Preliminaries and Notations Throughout this pap er, a capital letter (usually N ) is used to denote th e size of the vec tor, w h ic h is equiv alen t to the largest ind ex of a non-zero v alue, and a small letter (usually n ) is used to denote the num b er of n on-zero v alues. It is assumed that the v ectors are not giv en explicitly , rather th ey are giv en as a set of ( index, v al ue ) pairs, for all the non-zero v alues. A con volution uses t w o initial functions, v 1 and v 2 , to pr o d uce a third fun ction w . W e f ormally define a discrete con vo lution. Definition 1 L et V 1 b e a function wh ose domain is { 0 , ..., N 1 − 1 } and V 2 a function whose do main is { 0 , ..., N 2 − 1 } . We may v i ew V 1 and V 2 as arr ays of nu mb ers, whose lengths ar e N 1 and N 2 , r esp e ctively. The d iscrete conv olution of V 1 and V 2 is the p olynomial multipl ic ation W [ j ] = N 2 − 1 X i =0 V 1 [ j + i ] V 2 [ i ] . In the general case, the conv olution can b e computed by u s ing the F ast F our ier T ransform (FFT) [2]. This can b e done in time O ( N 1 log N 2 ), in a compu tatio nal m od el with word size O (log N 2 ). In the sparse ca se, many v alues of V 1 and V 2 are 0. T hus, t hey do not con tribute to the conv olution v alue. In our con v en tion, the num b er of n on-zero v alues of V 1 ( V 2 ) is n 1 ( n 2 ). Clearly , we can compute the con v olution in time O ( n 1 n 2 ). The question p osed b y Muth ukrishn an [3] is whether the con v olution can b e compu ted in time o ( n 1 n 2 ). Cole and Hariharan’s su ggestion w as to us e length r e duction . Supp ose w e can map all the n on - zero v alues in to a s maller vec tor, say of size O ( n 1 log n 1 ). Supp ose also that this mapping is alignmen t preservin g in the sense that applying the same transformation on V 2 will guarant ee that the alignments are preserve d. T hen we can simply map the the vec tors V 1 and V 2 in to the smaller v ectors and then use FFT for the co nv olutions on the smaller v ectors, ac hieving time O ( n 1 log 2 n 1 ). The problem is that to-date there is no known mapping with th at alignment preservin g prop er ty . Cole and Hariharan [1] suggested a randomized idea that an s w ers th e problem with high p robabilit y . The reason th eir algorithm is not deterministic is the follo wing: In their le ngth reduction phase, sev eral indices of non -zero v alues in the original v ector ma y b e mapp ed into the s ame index in th e reduced size v ector. If the index of only one non-zero v alue is mapped into an index in the reduced size v ector, then this ind ex is denoted as singleton and the non-zero v alue is said to app ear as a singleton . I f more then one non-zero v alue is ma pp ed in to the same i nd ex in the reduced size vec tor, then this index is denoted as multipl e . The m ultiple case is p roblematic since we ca n not b e s ure of the righ t alignmen t. F ortu nately , Cole and Hariharan sh ow ed a metho d wh ereb y in O (log n 1 ) tries, the pr ob ab ility that some index will always b e in a multiple situation is small. In [ ? ], a d eterministic solution to t he m ultiple problem w as presen ted. That solution utilized num b er th eoretic ideas. The new idea of this p ap er is to imp ro v e the redu ction size by using p olynomials to represent the lo cation of the non-0 elemen ts of the giv en v ectors. 3 3 The New Length Reduction T ec hnique for the P olynomial Case The pr op osed tec h nique d eals with the case that N 1 is p olynomial in n 1 , th u s the in dices are b ounded b y n c 1 . In the case where, N 1 is exp onentia l in n 1 , the r eduction to a p olynomial case can b e used. The main idea of the algorithm is to deriv e a set of unique polynomials from eac h non-zero in dex in V 1 , and one p olynomial for eac h n on -zero in V 2 . Eac h assignment for the p olynomials in F q , where q is a p r ime num b er of size Θ( n 1 ) w ill giv e a differen t mapping of the non-zeros in V 1 and in V 2 to v ectors of size q . Th e conv olution will b e p erf ormed b et wee n the vec tors obtained from V 1 and V 2 under the same assignments. The first step of the algorithm is to c ho ose a pr im e num b er of size Θ( n 1 ), and create a p olynomial for eac h non-ze ro index in V 1 . The created p olynomial of ind ex i w ill b e denoted as the base p olynomial of T [ i ]. The creation of the p olynomial is done by repr esen ting the index as a num b er in base ( q − 1) 2 . Eac h digit is interpreted as a co efficien t of the p olynomial. F or example: I f q = 13, then ind ex 95 in base 10 is 235 in b ase (13 − 1) 2 = 6 w hic h is repr esen ted by the p olynomial 2 X 2 + 3 X + 5. Since the indices in V 1 are b ounded b y n c 1 , and q is Θ( n 1 ), then the degree of the p olynomials whic h creat ed in this ste p is b ounded by c . In the n ext step, from eac h p olynomial w e create 2 c p olynomials. This is done b y giving to c hoices f or eac h co efficien t o f the p olynomial: (1) Lea v e it as is. (2) Add ( q − 1) 2 to the co efficien t and decrease by 1 the co efficien t of the higher degree. W e d o this for all the co efficien ts of the p olynomial except for the co efficien t of the highest degree. Example 1 Supp ose we have a non-zer o i ndex 95 , using q = 13 we get the b ase p olynomial 2 X 2 + 3 X + 5 . A f ter the se c ond step we w il l obtain 4 p olynomials: 2 X 2 + 3 X + 5 , 2 X 2 + 2 X + 11 , X 2 + 9 X + 5 , X 2 + 8 X + 11 . The first p olynomial is the b ase p olynomia l. The se c ond p olynomial was obtaine d by adding 6 to the fir st c o efficie nt and de cr e asing the se c ond c o efficient by one. The 3r d and the 4th p olynomials wer e cr e ate d by ad ding 6 to the se c ond c o efficient of the first an d se c ond p olynomials r esp e ctively, and de cr e asing the thir d c o efficient by one. The duplication of the p olynomials w as mad e to meet the distance pr eserving requiremen t from the length reduction sp ecified in the follo win g Lemma: Lemma 1 F or any assignment of X , if V 2 [0] is aligne d with the b ase p olynomial r epr esenting V 1 [ i ] , then V 2 [ j ] w il l b e aligne d w ith o ne of th e p olynomial s r epr esenting V 1 [ i + j ] . Pro of: Let q b e the c h osen prime num b er. Index 0 in V 2 is represent ed b y the p olynomial 0, and index j in V 2 is represented by the a p olynomial A = a c X c + a c − 1 X c − 1 + ... + a 0 . In dex i in V 1 is represent ed by a p olynomial of the form B = b c X c + b c − 1 X c − 1 + ... + b 0 , and in dex i + j in V 1 is represent ed b y a p olynomial D = d c X c + d c − 1 X c − 1 + ... + d 0 . Note that the co efficien ts a i and b i are smaller then ( q − 1) 2 . Clearly , if V 2 [0] is aligned with V [ i ], then for an y assignmen t o f X , V 2 [ j ] will b e aligned with the p olynomial A + B = ( a c + b c ) X c + ( a c − 1 + b c − 1 ) X c − 1 + ... + ( a 0 + b 0 ). No w lets lo ok at the 4 first coefficient of D , sin ce a 0 and b 0 are smaller then ( q − 1) 2 , then there are only tw o cases: (1) ( a 0 + b 0 ) < ( q − 1) 2 , th us d 0 = a 0 + b 0 . (2) ( a 0 + b 0 ) > = ( q − 1) 2 , th us d 0 = a 0 + b 0 − ( q − 1) 2 whic h is co vered by the p olynomial w h ere ( q − 1) 2 w as added to the fi rst co efficien t. In the later case, one wa s added to the second co efficien t, thus w e decrease the n ext co efficien t whenev er w e add ( q − 1) 2 to th e cur ren t co efficien t. The same cases exi st also in all the co efficien t, but a p olynomial w as created for eac h p ossible case (2 c cases), thus one of the created p olynomials will b e equ al to th e p olynomial A + B . ⊓ ⊔ Note that all the 2 c × n 1 created p olynomials are u nique, and in F q . Assigning a v alue to the p olynomials in F q will give a v ector of size q . Lemma 2 Any two p olynomials c an b e mapp e d to the same lo c ation in at most c a ssignments. Pro of: The distance b et we en an y t wo p olynomials giv es a p olynomial, wh ere the degree of th e difference p olynomial is b ounded by c . Since b oth p olynomials giv e the same index und er the selected assignmen t, then the assigned v alue is a ro ot of the difference p olynomial. The degree of this p olynomial is b ound ed by c , thus it can hav e at most c different ro ots in F q . ⊓ ⊔ Since any p olynomial can b e mapp ed in to the same lo cation with at most 2 c × n 1 − 1 other p olynomials, a nd with eac h of them at most c times, due to Lemma 2, then w e get the follo wing Corollary: Corollary 1 Any p olynomial c an app e ar as a multiple i n not mor e then c × 2 c × n 1 ve ctors. The last step of the length reduction algorithm is to fi nd a set of O (log n 1 ) assignments which will ensure that eac h p olynomial will app ear as a singleton at least once. The selection of the O (log n 1 ) assignmen ts is done as follo w s: C on s truct table A with 2 c × n 1 columns and c × 2 c +1 × n 1 ro ws. Ro w i corr esp ond to an assigned v alue a i and the corresp onding reduced length v ector V 1 ,i . A column corresp ond s to a p olynomial P j . T he v alue of A ij is set to 1 if p olynomial j app ears as a singleton in vect or V 1 ,i . Due to Corollary 1, the num b er of zeros in eac h column can not exceed c × 2 c × n 1 . Thus, in eac h column there are 1’s in at least half of th e ro ws, which means that the table is at least half full. Since the table is a t least half f ull there exists a ro w in wh ic h th er e is one in at least half of the columns. The assignment v alue whic h generated this r o w is c h osen, and all the columns where t here w as a 1 in the selected ro w are d eleted from the table. Recursiv ely an other assignment v alue is chosen and the table size is halv ed again, until all the columns are deleted. Since at eac h step at least half of the columns are deleted, the n umb er of prime num b er c hosen can n ot exceed log (2 c × n 1 ) = c log n 1 . Time: Creating vect or V 1 ,i (ro w i ) ta ke s O ( n 1 ) time. Since w e start with a full matrix of O ( n 1 ) ro ws then the initializatio n tak es O ( n 2 1 ) time. Cho osing the O (log n 1 ) assignmen t v alues is done recursiv ely . Th e recurr ence is: t ( n 2 1 ) = n 2 1 + t ( n 2 1 2 ) The closed form of this r ecurrence is O ( n 2 1 ). 5 4 The New Algorithm for The Exp onen tial Case In this case, as prop osed in [ ? ], eac h of the v ectors V 1 and V 2 is reduced in to a single v ector of size O ( n 4 1 ), where all the non-zeros app ear as singletons. The reduction is preformed using the mod ulus function with a pr im e num b er q of s ize O ( n 4 1 ). It was already pr o ven there th at there are at most n 3 1 prime n u m b er o f size O ( n 4 1 ), whic h generate at least one multiple. Thus, b y testing n 3 1 + 1 prime n umb ers we ensur e that at least one of them pr o d uce a v ector w ith n o multiples. In order to find such a p rime num b er, w e fin d n 3 1 + 1 of size O ( n 4 1 ). Then w e m ultiply all the prime num b ers to rece ive a large n umber Q . In add ition w e ha ve at most n 2 1 differen t distances b et w een an y t wo non-zeros. W e m ultiply all of them to receiv e the large num b er D . The n ext step is to fin d the greatest common d ivider ( GC D ) b et w een Q and D . Since there is at least one pr ime n umb er in Q whic h d o es not divid e D , then GC D ( Q, D ) is less then Q . D ividing Q b y the GC D ( Q, D ) will giv e P whic h is the multiplica tion of all the prime num b ers that create only sin gleto ns. The last step is to find at least one of them. This is done using a binary search on the prime num b ers. W e tak e the m ultiplication of half o f the p rime n umb ers Q ′ , and fi nd the GC D ( Q ′ , P ). If GC D ( Q ′ , P ) > 1 w e cont inue with this set of prim e num b ers and m ultiply half of them it erativ ely . O therwise, we contin ue with the other half o f the prime num b ers. After O (log n 1 ) iterations we will find one p rime num b er whic h will generate only singletons. The algorithm app ears in detail b elow. Algorithm – N 1 is exp onen tial in n 1 1. F i nd n 3 1 + 1 p rime numb ers of size O ( n 4 1 ) . 2. M ultiply all t he prime numb ers t o obtain Q . 3. M ultiply all t he difference b et w een a n y tw o non-zero indices to obta i n D . 4. S et P = Q GC D ( Q,D ) . 5. Let S b e the set of al l prime numb ers. 6. Wh i le the siz e of S is larger then 1 do: (a) Let S ′ b e a set of the first ha lf of prime numb ers i n S . (b) S et Q ′ to b e the multip l ication of all the prime numb ers in S ′ . (c) If GC D ( Q ′ , P ) > 1 then set S = S ′ , otherwise set S = S/S ′ . end Algorithm Correctness: Im mediately follo ws from the discus s ion. Time: Step 1 is p erformed in time O ( n 3 1 p olylog( n 1 )) using the p rimalit y testing describ ed in [ ? ]. Step 2 is d on e by building a b inary tree of multi plication wh ere eac h n o d e con tain the multiplica tion of the t wo num b er in the low er leve l. This tree has O (log n 1 ) lev els. In the lea ve s there are n 3 1 prime n umbers with log n 1 bits, so the total n umb er of bits in eac h lev el is O ( n 3 1 log n 1 ). A m ultiplication of tw o n u m b ers can b e compu ted in time O ( b log b log log b ) [4], where b is the 6 n umb er of bits. T h us eac h leve l can b e computed in time O ( n 3 1 p olylog( n 1 )) and the total time for step 2 is O ( n 3 1 p olylog( n 1 )). step 3 is preformed in the same w a y , but this time in the lea v es there are n 2 1 n umb ers with n 1 bits, thus eac h le ve l has n 3 1 bits and the time for t his step is O ( n 3 1 log n 1 ). In step 4 we calculate the GC D of t wo n umb ers with O ( n 3 1 log n 1 ) bits. Th is can b e calculated in time O ( n 3 1 p olylog( n 1 )) using [ ? ]. The ca lculation for step 6( b) was already perf orm ed in step 2, and step 6(c) can b e calculated in time O ( n 3 p olylog( n 1 )), th us the time of step 6 is O ( n 3 1 p olylog( n 1 )). F ollo win g this discussion the total time of this algorithm is O ( n 3 1 p olylog( n 1 )). 5 Conclusion and Op en P roblems Improv ed deterministic algorithms f or L en gth Reduction and S parse Conv olution wh ere pr esen ted in this pap er. Th ese can b e u sed as to ols to pr o vid e faster algorithms for s ev eral well k n o w n problems. The deterministic time ac h iev ed for con v olving inpu t patterns with a fixed text is the same as the b est known rand omized algorithm. An imp ortant pr oblem remains: Can the Lengt h Reductio n and Sparse Conv olution problems b e solv ed in real time without th e need of the prepro cessing step, or a lternately , can the p repro cessing time b e reduced from quadr atic? References [1] R. Co le and R. Hariharan. V erifying c a ndidate matc hes in sparse and wildcard matching. In Pr o c. 34st Annual Symp osium on the The ory of Computing (STOC) , pa g es 592–601, 2 0 02. [2] T. H. Cormen, C. E. Leiserson, and R. L. Rivest. Intr o duction to Algorithms . MIT Press and McGraw- Hill, 1992. [3] S. Muthukrishnan. New results and open problems related to non-standard stringolo gy . In Pr o c. 6th Combinatorial Pattern Matching Confe r enc e , pages 298–317. Lecture No tes in Computer Science 9 37, Springer-V erlag, 1995. [4] A. Sch¨ onhage and V. Strasse n. Schnelle m ultiplik ation gros s er zahlen. Computing (Ar ch. Elektr on. R e chnen) , 7:281 –292, 19 71. MR 45 # 1431. 7