Two sources are better than one for increasing the Kolmogorov complexity of infinite sequences

Tw o sources are b etter than one fo r increasing the Kolmogoro v complexit y of infinite sequences Marius Zimand ∗ Departmen t of Computer and Information Sciences T o wson Univ ersit y Abstract The randomness rate of an infinite binary sequence is characterized b y the sequence of ratios b et ween the Ko lmogorov complexit y and the length of the initial segments of the sequence. It is kno wn that there is no unifor m effectiv e pro cedure that transforms one input sequence int o a no t her sequence with higher rando mness ra te. By contrast, w e display such a uniform effective pro cedure having as input t wo indep endent seq uences with pos itive but arbitrar ily small co nstant ra ndomness rate. Moreover the transformatio n is a truth-table reduction and the output has randomness ra te a rbitrar ily close to 1. Key words: Kolmogoro v complexit y , Hausd orff dimension. 1 In tro duction It is a b asic fact that no fun ction can increase the amount of randomness (i.e. , en tropy) of a fi nite stru cture. F ormally , if X is a distrib ution on a fi nite set A , then for any fu nction f mapping A int o A , the (Shann on) entrop y of f ( X ) cannot b e larger than th e ent ropy of X . As it is usually th e case, the ab ov e fact has an analogue in algorithmic information theory: for any finite binary string x and an y co mpu table function f , K ( f ( x )) ≤ K ( x ) + O (1), where K ( x ) is the Kolmogoro v complexit y of x and the constan t dep ends only on the u n derlying univ ersal mac hine. The ab o ve inequalit y h as an immediate one-line pro of, but the analoguous s tatement when we m o v e to infinite sequences is not kno wn to hold. F or an y σ ∈ [0 , 1 ], we sa y that an infinite binary sequence x has ran d omness rate σ , if K ( x (1 : n )) ≥ σ n for all sufficien tly large n , wh ere x (1 : n ) denotes the initial segmen t of x of length n . 1 The question b ecomes: if x h as randomn ess rate 0 < σ < 1, is there an effectiv e transform ation f suc h that f ( x ) has randomness rate greater than that of x ? Unlik e the case of finite strings, infinite sequences with p ositiv e randomn ess rate p ossess an infinite amoun t of r an d omness (ev en though it is sparsely distributed) and thus it cannot b e ruled out that there ma y b e a w a y to concen trate it and obtain a sequence with higher randomness rate. This is a natural question, first raised by Reimann [Rei04], which has receiv ed significant atten tion r ecen tly (it is Question 10.1 in the list o f open qu estions of Miller and Nies [M N06 ]). So far, there exist sev eral partial results, mostly negativ e, obtained by restricting the t yp e of transformation. Reimann and T erwijn [Rei0 4 , Th 3.10] ha v e sho wn that for eve ry constan t c < 1, there exists a sequence x su c h th at if f is a m an y-one r eduction, then the randomness rate ∗ The author is sup p orted by NSF grant CCF 0634830. Part of this work w as done while visiting Un ivers ity of Auckland, New Zealand. 1 The rand omness rate of x is very close to the notion of constructive Hausdorff d imension of x [Lut03 , May02, Rya84, Sta05]; how ever since this p ap er is ab out han d ling rand omn ess and not ab out measure-theoretical issues w e p refer the randomness terminology . 1 of f ( x ) cannot b e larger than c . This result has b een impro v ed by Nie s and Reimann [NR06] to w tt-redu ctions. More pr ecisely , they sho we d that for all rational c ∈ (0 , 1), there exists a sequence x with randomness rate c such that f or all wtt-reductions f , f ( x ) has rand omness rate ≤ c . Bien v en u, Dot y , and Stephan [BDS07] ha v e obtained an imp ossibilit y result for the general case of T uring redu ctions, whic h, ho w ev er, is v alid on ly for uniform reductions. Building on the result of Nies and Reimann, they sho w that for every T uring reduction f and all constan ts c 1 and c 2 , w ith 0 < c 1 < c 2 < 1 , there exists x with rand omness rate ≥ c 1 suc h that f ( x ), if it exists, has randomn ess rate < c 2 . In other w ord s , lo osely sp eaking, no effectiv e uniform transformation is able to raise the randomness rate from c 1 to c 2 . Th us the question “Is there an y effectiv e transformation that on input σ ∈ (0 , 1], ǫ > 0, and x , a sequen ce with randomness rate σ , pro duces a string y with randomness rate σ + ǫ ?” has a nega tiv e answ er. On the p ositiv e side, Dot y [Dot07] h as shown that f or eve ry constant c th ere exists a un iform effectiv e trans formation f able to transform an y x with rand omness rate c ∈ (0 , 1] in to a sequence f ( x ) that, for infinitely many n , has the initial segments of length n with Kolmogoro v complexit y ≥ (1 − ǫ ) n (see Dot y’s pap er for the exa ct statemen t). Ho w ev er, since Dot y’s transformation f is a wtt-reduction, it follo ws from Nies and Reimann’s result that f ( x ) also has infinitely man y initial segmen ts with no increase in the Kolmog oro v complexity . In the case of finite strin gs, as w e ha ve observ ed earlier, there is no effectiv e transformation that increases the abs olute amount of Kolmogoro v complexit y . Ho w ev er, some positive resu lts do exist. Buh rman, F ortnow, Newman, and V ereshc hagin [BFNV05] show that, for an y non- random string of length n , one can flip O ( √ n ) of its bits and obtain a string w ith higher Kolmogoro v complexit y . F ortno w, Hitc hco c k, P a v an, Vino d c handran, and W ang [FHP + 06] sho w th at for any 0 < α < β < 1, there is a p olynomial-time pro cedure that on in put x w ith K ( x ) > α | x | , u sing a constan t num b er of advice b its (which d ep end on x ), builds a string y with K ( y ) ≥ β | y | and y is sh orter than x by only a multiplica tiv e constan t. Our main r esult concerns infinite sequences and is a p ositiv e one. Recall that Bien ve nu, Dot y and Stephan hav e s h o wn that there is no uniform effectiv e w a y to increase the randomn ess rate when the inp ut consists of one sequ en ce w ith positive rand omness rate. W e show that if instead the input consists of two suc h sequences that are indep endent, then suc h a unif orm effectiv e transformation exists. Theorem 1.1 (Main R esult) Ther e exists an effe ctive tr ansformation f : Q × { 0 , 1 } ∞ × { 0 , 1 } ∞ → { 0 , 1 } ∞ with the fol lowing pr op erty: If th e input is τ ∈ (0 , 1] and two indep en- dent se que nc es x and y with r andomness r ate τ , then f ( τ , x, y ) has r andomness r ate 1 − δ , for al l δ > 0 . M or e over, the effe ctive tr ansforma tion is a truth-table r e duction. Effectiv e tran s formations are essen tially T uring reductions that are u niform in the p arameter τ ; see Section 2. Tw o sequences are ind ep end ent if they d o not con tain m uch common inf orm ation; see Section 3. One key elemen t of the pr o of is in spired from F ortno w et a l.’s [FHP + 06], wh o show ed that a rand omness extractor can b e u sed to construct a pro cedu re that increases the Kolmogoro v complexit y of fin ite strings. Their pro cedure for in creasing the Kolmogoro v complexit y runs in p olynomial time, but uses a small amount of advice. T o obtain the p olynomial-time efficiency , they h ad to use the m u lti-source extrac tor of Barak, Impagliaz zo, and Wigderson [BIW0 4], whic h requires a num b er of sources that is dep endent on the in itial min-entrop y of the sources and on the desired qualit y of the output. In our case, we are not concerned ab out the efficiency of the transformation (this of course simplifies our task), b u t, on the other hand, we wan t it completely effectiv e (with no advice), w e wan t it to w ork with just t wo sources, and we w an t it to hand le infi nite s equences. In p lace of an extractor, we provide a p ro cedure with similar functionalit y , using the probabilistic metho d whic h is next derandomized in the trivial wa y b y brute force sea rching. Since w e handle infinite sequences, w e hav e to iterate the procedu re 2 infinitely man y times on fi nite blo c ks of the tw o sou r ces and this necessitates solving some tec h nical issues related to the indep endence of the bloc ks. 2 Preliminaries W e w ork o v er the binary alph ab et { 0 , 1 } . A string is an elemen t of { 0 , 1 } ∗ and a sequence is an elemen t of { 0 , 1 } ∞ . If x is a string, | x | d enotes its length. If x is a string or a sequence and n, n 1 , n 2 ∈ N , x ( n ) d enotes the n -th bit of x and x ( n 1 : n 2 ) is the s ubstring x ( n 1 ) x ( n 1 + 1) . . . x ( n 2 ). The cardin alit y of a fin ite set A is d enoted k A k . Let M b e a standard T uring mac hine. F or an y string x , define the (plain) Kolmo gor ov c omplexity of x with resp ect to M , as K M ( x ) = min {| p | | M ( p ) = x } . There is a univ ersal T uring mac hine U suc h that for ev ery mac hine M there is a constant c suc h that for all x , K U ( x ) ≤ K M ( x ) + c. (1) W e fix such a univ ersal mac hine U and dropp in g the sub script, we let K ( x ) d enote the Kol- mogoro v complexit y of x with r esp ect to U . F or the concept of conditional K omogoro v com- plexit y , the underlying machine is a T ur ing machine that in addition to the read/w ork tap e whic h in the initial state con tains the inp ut p , has a second tap e con taining initially a string y , whic h is called the conditioning information. Giv en su c h a m ac hine M , w e defi n e the Kolmogoro v complexit y of x conditioned b y y with resp ect to M as K M ( x | y ) = min {| p | | M ( p, y ) = x } . Similarly to the ab o v e, there exist un iv ersal m ac hines of this t yp e and they satisfy the relation similar to Equation 1 , bu t for conditional complexit y . W e fix su c h a u niv ersal mac hin e U , and dropping the subscript U , w e let K ( x | y ) denote the Kolmogoro v complexit y of x conditioned b y y with resp ect to U . W e briefly use th e concept of pr efix- fr e e c omplexity , whic h is d efined similarly to plain Kolmogoro v complexit y , th e difference b eing th at in th e case of prefix-free complexit y the domain of the u nderlying mac hines is required to b e a pr efi x-free set. Let σ ∈ [0 , 1]. A sequence x has rand omness rate σ if K ( x (1 : n )) ≥ σ · n , for almost ev ery n (i.e., the set of n ’s violating the inequalit y is finite). An effectiv e transformation f is repr esen ted b y a t w o-oracle T ur in g mac hine M f . The mac hine M f has acce ss to t w o oracles x and y , w hic h are binary sequences. When M f mak es the qu ery “ n -th b it of firs t oracle?” (“ n -th bit of second oracle?”), th e mac hine obtains x ( n ) (resp ectiv ely , y ( n )). On input ( τ , 1 n ), where τ is a rational (giv en in some canonical represen- tation), M f outputs one bit. W e sa y th at f ( τ , x, y ) = z ∈ { 0 , 1 } ∞ , if for all n , M f on input ( τ , 1 n ) and w orking with oracles x and y halts and outputs z ( n ). (Effectiv e transformations are more commonly called T ur ing reductions. If τ w ould b e em b edd ed in th e machine M f , instead of b eing an inpu t, w e w ould sa y that z is T uring-redu cible to ( x, y ). Our appr oac h emphasizes the fact th at we wan t a f amily of T uring redu ctions that is uniform in th e parameter τ .) In case the mac hin e M f halts on all inputs and w ith all oracles, we sa y that f is a truth-table reduction. 3 Indep endence W e n eed to requir e th at the tw o inputs x and y that app ear in th e main result are really distinct, or in th e algorithmic-i nform ation theoretical terminology , indep endent . 3 Definition 3.1 Two infinite binary se quenc es x , y ar e indep endent if f or al l natur al numb e rs n and m , K ( x (1 : n ) y (1 : m )) ≥ K ( x (1 : n )) + K ( y (1 : m )) − O (log( n ) + log ( m )) . The definition sa ys that, mo dulo additiv e logarithmic terms, there is no shorter w ay to describ e the concatenation of any t wo initial s egments of x and y than ha ving the information that describ es the initial segmen ts. It can be sho w n that th e fact that x and y are indep end ent is equ iv alen t to sa ying th at for ev ery natural n umb ers n and m , K ( x (1 : n ) | y (1 : m )) ≥ K ( x (1 : n )) − O (log ( n ) + log( m )) . (2) and K ( y (1 : m ) | x (1 : n )) ≥ K ( y (1 : m )) − O (log ( n ) + log ( m )) . (3) Th us, if t wo sequences x and y are indep endent, no initial segmen t of one of the sequence can help in getti ng a shorter description of an y initial segment of the ot her sequ ence, mo dulo additiv e logarithmical terms. In our main result, the inp ut consists of t w o sequences x and y that are ind ep endent and that ha v e Kolmogoro v rate σ for some p ositive constant σ < 1. W e sket c h an argument showing that suc h sequences exist. In our sk etc h w e tak e σ = 1 / 2. W e start with an arb itrary random (in the Martin-L¨ of sense) sequence x . Next using the m ac hinery of Martin-L¨ of tests relativized with x we infer the existence of a sequen ce y that is random relativ e to x . F rom the theory of Martin-L¨ of tests, w e deduce that there exists a co nstant c s uc h that for all m , H ( y (1 : m ) | x ) ≥ m − c , where H ( · ) is the prefix- free ve rsion of complexit y . Since H ( y (1 : m )) ≤ m + O (log m ), for all m , we conclude that H ( y (1 : m ) | x ) ≥ H ( y (1 : m )) − O (log m ), for all m . Therefore, H ( y (1 : m )) | x (1 : n )) ≥ H ( y (1 : m ) | x ) − O (log n ) ≥ H ( y (1 : m )) − O (log n + log m ), for all n and m . S ince the prefix-free complexity H ( · ) and the plain complexit y K ( · ) are within O (log m ) of eac h other, it follo ws that K ( y (1 : m )) | x (1 : n )) ≥ K ( y (1 : m )) − O (log n + log m )), for all n and m . Th is implies K ( x (1 : m ) y (1 : n )) ≥ K ( x (1 : n )) + K ( y (1 : m )) − O (log ( n ) + log ( m )), for all n, m . Next we construct x ′ and y ′ b y in serting in x and resp ectiv ely y , the bit 0 in all ev en p ositions, i.e., x ′ = x 1 0 x 2 0 . . . (where x i is the i -th bit of x ) an d y ′ = y 1 0 y 2 0 . . . . Clearly , K ( x (1 : n )) and K ( x 1 0 . . . x n 0) a re within a co nstant of eac h other, and the same holds for y and y ′ . It f ollo w s that x ′ and y ′ are indep endent and ha v e rand omn ess rat e 1 / 2. 4 Pro of of Main Result 4.1 Pro of Ov erview W e present in a simp lifi ed setting th e main ideas of the constru ction. Supp ose we ha v e tw o indep end en t str in gs x and y of length n such that K ( x ) = σ n and K ( y ) = σ n , for some σ > 0. W e wan t to construct a str ing z of length m s uc h that K ( z ) > (1 − ǫ ) m . The key idea (b orrow ed from the theory o f randomness extractors) is to use a fun ction E : { 0 , 1 } n × { 0 , 1 } n → { 0 , 1 } m suc h that eve ry large enough rectangle of { 0 , 1 } n × { 0 , 1 } n maps ab out the same num b er of pairs in to all elements of { 0 , 1 } m . W e say that su c h a f unction is r e gular (the formal Definition 4.8 has some p arameters whic h quanti fy the degree of regularit y). T o illustrate the idea, sup p ose for a momen t that w e ha ve a function E : { 0 , 1 } n × { 0 , 1 } n → { 0 , 1 } m that, for all subsets B ⊆ { 0 , 1 } n with k B k ≈ 2 σn , has the prop ert y that any a ∈ { 0 , 1 } m has the same num b er of preimages in B × B , w hic h is of course k B × B k / 2 m . Then for any A ⊆ { 0 , 1 } m , E − 1 ( A ) ∩ ( B × B ) h as size 4 k B × B k 2 m · k A k . Let us tak e z = E ( x, y ) and let us supp ose that K ( z ) < (1 − ǫ ) m . Note that the set B = { u ∈ { 0 , 1 } n | K ( u ) = σ n } h as size ≈ 2 σn , the set A = { v ∈ { 0 , 1 } m | K ( v ) < (1 − ǫ ) m } has size < 2 (1 − ǫ ) m and that x and y are in E − 1 ( A ) ∩ ( B × B ). By the ab o v e observ ation the set E − 1 ( A ) ∩ ( B × B ) has size ≤ 2 σ n · 2 σ n 2 ǫm . Since E − 1 ( A ) ∩ ( B × B ) ca n b e e numerated effectiv ely , an y pair of strings in E − 1 ( A ) ∩ B × B can b e describ ed b y its rank in a fixed en umeration of E − 1 ( A ) ∩ B × B . In particular ( x, y ) is suc h a pair and therefore K ( xy ) ≤ 2 σ n − ǫm . On the other hand, since x and y are indep endent , K ( xy ) ≈ K ( x ) + K ( y ) = 2 σ n . Th e con tradiction w e ha v e reac hed shows that in fact K ( z ) ≥ (1 − ǫ ) m . A fu nction E having th e strong regularit y requiremen t stated ab o v e ma y not exist. F ortu- nately , using the probab ilistic metho d, it can b e shown (see Section 4.3) that, for all m ≤ n 0 . 99 σ , there exist a fun ction E : { 0 , 1 } n × { 0 , 1 } n → { 0 , 1 } m suc h that all str in gs a ∈ { 0 , 1 } m ha v e at most 2( k B × B k / 2 m ) preimages in any B × B as ab o v e (instead of ( k B × B k / 2 m ) preimages in the ideal, bu t n ot realizable, setting we us ed ab ov e). Once w e know that it exists, su c h a fun ction E can b e fou n d effectiv ely b y exhaustive searc h. Then the argument ab o v e, with some minor modifications, go es through. In fact, when we app ly this idea , we only kno w that K ( x ) ≥ σ n and K ( y ) ≥ σ n and therefore w e need the fun ction E to satisfy a stronger v arian t of regularit y . Ho wev er, the main idea r emains the same. Th us there is an effectiv e wa y to p r o duce a string z with K olmogoro v complexit y (1 − ǫ ) m from tw o indep endent strings x an d y of length n and with K olmogoro v co mp lexity σ n . Recall that, in f act, the input consists of tw o indep en den t infinite sequences x and y with ran d omness rate τ > 0. T o tak e adv an tage of the pro cedure sk etc h ed ab ov e whic h w orks for finite strings, w e sp lit x and y in to fin ite strings x 1 , x 2 , . . . , x n , . . . , and resp ectiv ely y 1 , y 2 , . . . , y n , . . . , su c h that the blo cks x i and y i , of length n i , ha ve still enough Kolmogo rov complexit y , sa y ( τ / 2) n i , conditioned b y the previous bloc ks x 1 , . . . , x i − 1 and y 1 , . . . , y i − 1 . The splitting of x and y into blo c ks and the prop erties of th e b lo c ks are present ed in Section 4.2. T hen using a regular function E i : { 0 , 1 } n i × { 0 , 1 } n i → { 0 , 1 } m i , we build z i = E i ( x i , y i ). By modifyin g sligh tly the argumen t describ ed a b o v e, it can b e sho wn that K ( z i | x 1 , . . . , x i − 1 , y 1 , . . . , y i − 1 ) > (1 − ǫ ) m i , i.e., z i has high Kolmogoro v complexit y even conditioned by the previous b lo c ks x 1 , . . . , x i − 1 and y 1 , . . . , y i − 1 . It follo ws that K ( z i | z 1 , . . . , z i − 1 ) is also close to m i . W e fi nally tak e z = z 1 z 2 . . . , and u sing the ab ov e pr op ert y of eac h z i , w e in fer that f or every n , the prefix of z of length n h as randomn ess rate > (1 − ǫ ) n . I n other w ords, z has randomness r ate (1 − ǫ ), as desired. 4.2 Splitting t he tw o inpu ts The t wo inpu t sequ ences x and y fr om T heorem 1.1 are broke n into fin ite blo c ks x 1 , x 2 , . . . , x i , . . . and resp ectiv ely y 1 , y 2 , . . . , y i , . . . . T h e division is done in su c h a manner t hat x i (resp ectiv ely , y i ) h as high Komogoro v complexit y rate cond itioned b y the pr evious blo cks x 1 , . . . , x i − 1 (re- sp ectiv ely b y th e blo cks y 1 , . . . , y i − 1 ). Th e follo win g lemma shows h ow this division is done. Lemma 4.1 (Splitting lemma) L et x ∈ { 0 , 1 } ∞ with r andomness r ate τ , for some c onstant τ > 0 . L et 0 < σ < τ . F or any n 0 sufficiently lar ge, ther e is n 1 > n 0 such that K ( x ( n 0 + 1 : n 1 ) | x (1 : n 0 )) > σ ( n 1 − n 0 ) . F urthermor e, ther e is an effe ctive pr o c e dur e that on input n 0 , τ and σ c alculates n 1 . Pro of . Let σ ′ b e suc h that 0 < σ ′ < τ − σ . T ak e n 1 =  1 − σ σ ′  n 0 . 5 Supp ose K ( x ( n 0 + 1 : n 1 ) | x (1 : n 0 )) ≤ σ ( n 1 − n 0 ). Then x (1 : n 1 ) can be reco nstru cted from: x (1 : n 0 ), the description of x ( n 0 + 1 : n 1 ) give n x (1 : n 0 ), n 0 , extra constan t num b er of bits describing the pro cedure. So K ( x (1 : n 1 )) ≤ n 0 + σ ( n 1 − n 0 ) + log n 0 + O (1) = σ n 1 + (1 − σ ) n 0 + log n 0 + O (1) ≤ σ n 1 + σ ′ n 1 + log n 0 + O (1) < τ n 1 (if n 0 suffic. large) , (4) whic h is a co ntradictio n if n 1 is sufficien tly large. No w w e define the p oin ts where w e split x and y , the t w o sources. T ak e a , the p oint fr om where the S plitting Lemma h olds. F or the rest of th is section we consider and b =  1 − σ σ ′  . The follo wing sequence repr esen ts the cutting p oin ts that will define the blo c ks. It is defined recursiv ely , as follo w s: t 0 = 0, t 1 = a , t i = b ( t 1 + . . . + t i − 1 ). It can b e seen that t i = ab (1 + b ) i − 2 , for i ≥ 2. Finally , we define the blo cks: for eac h i ≥ 1, x i := x ( t i − 1 + 1 : t i ) and y i = y ( t i − 1 + 1 : t i ), and n i := | x i | = | y i | = ab 2 (1 + b ) i − 3 (the last equalit y h olds for i ≥ 3). W e also denote by ¯ x i the concatenati on o f the blo c ks x 1 , . . . , x i and by ¯ y i the concatenati on of the blocks y 1 , . . . , y i . Lemma 4.2 1. K ( x i | ¯ x i − 1 ) > σ n i , for al l i ≥ 2 (and the analo gue r e lation holds for the y i ’s). 2. log | x i | = Θ( i ) and log | ¯ x i | = Θ( i ) , for al l i (and the analo gue r elation holds for th e y i ’s). Pro of . The first p oint follo ws fr om th e S plitting Lemma 4.1, and th e second p oin t follo ws immediately f rom the definition of n i (whic h is the length of x i ) and of t i (whic h is the length of ¯ x i ). The follo w ing facts state some basic algorithmic-information theoretical prop erties of the blo c ks x 1 , x 2 , . . . . and y 1 , y 2 , . . . . W e fi rst recall the follo wing b asic fact (for example, see Alexander S hen’s lecture notes [She00]). Theorem 4.3 F or al l finite binary strings u and v , (i) K ( v u ) ≤ K ( u ) + K ( v | u ) + O (log K ( u ) + log K ( v )) . (ii) K ( vu ) ≥ K ( u ) + K ( v | u ) − O (log K ( u ) + log K ( v )) . The hidden c onstants dep end only on the unive rsal machine that defines the c omplexity K ( · ) . Lemma 4.4 F or al l finite binary strings u and v ,   K ( v | u ) −  K ( v u ) − K ( u )    < O (log | u | + log | v | ) . Pro of . T heorem 4.3 implies   K ( v | u ) −  K ( v u ) − K ( u )    < O (log K ( u ) + log K ( v )). S ince K ( u ) ≤ | u | + O (1) and K ( v ) ≤ | v | + O (1), the conclusion follo ws. Lemma 4.5 F or al l i and j , (a)   K ( ¯ y i ¯ x j ) −  K ( ¯ y i ) + K ( ¯ x j )    < O ( i + j ) . (b)   K ( ¯ x i ¯ y j ) −  K ( ¯ x i ) + K ( ¯ y j )    < O ( i + j ) . 6 Pro of . W e pro ve ( a ) (( b ) is similar). K ( ¯ y i ¯ x j ) ≤ K ( ¯ y i ) + K ( ¯ x j ) + O (log ( K ( ¯ y i ) + log( K ( ¯ x j )) ≤ K ( ¯ y i ) + K ( ¯ x j ) + O (log | ¯ y i | + log | ¯ x j | ) = K ( ¯ y i ) + K ( ¯ x j ) + O ( i + j ) . (5) The fir st line follo ws from Theorem 4.3 (i) (keeping in mind th at K ( v | u ) ≤ K ( v ) + O (1)). F or the last line w e to ok in to accoun t that log | ¯ x j | = O ( j ) and log | ¯ y i | = O ( i ). On the other hand, K ( ¯ y i ¯ x j ) ≥ K ( ¯ y i ) + K ( ¯ x j ) − O (log | ¯ y i | + log | ¯ x j | ) = K ( ¯ y i ) + K ( ¯ x j ) − O ( i + j ) . (6) The first line f ollo w s from the indep end en ce of x and y . Com bining equations (5) and (6), t he conclusion follo ws. Lemma 4.6 F or al l i and j , (a)   K ( x i | ¯ x i − 1 ¯ y j ) − K ( x i | ¯ x i − 1 )   < O ( i + j ) . (b)   K ( y i | ¯ x j ¯ y i − 1 ) − K ( y i | ¯ y i − 1 )   < O ( i + j ) . Pro of . W e pro ve ( a ) (( b ) is similar). W e fir st ev aluate K ( x i | ¯ x i − 1 ¯ y j ). F rom Lemma 4.4,   K ( x i | ¯ x i − 1 ¯ y j ) −  K ( x i ¯ x i − 1 ¯ y j ) − K ( ¯ x i − 1 ¯ y j )    < O ( i + j ) . (7) It is easy to c h ec k that K ( x i ¯ x i − 1 ¯ y j ) is within O ( i ) f rom K ( ¯ x i − 1 x i ¯ y j ). Th us, w e ca n substitute K ( x i ¯ x i − 1 ¯ y j ) b y K ( ¯ x i ¯ y j ) and o btain,   K ( x i | ¯ x i − 1 ¯ y j ) −  K ( ¯ x i ¯ y j ) − K ( ¯ x i − 1 ¯ y j )    < O ( i + j ) . (8) Next, by Lemma 4.5,   K ( ¯ x i ¯ y j ) −  K ( ¯ x i ) + K ( ¯ y j )    < O ( i + j ) and   K ( ¯ x i − 1 ¯ y j ) −  K ( ¯ x i − 1 ) + K ( ¯ y j )    < O ( i + j ). Plugging these inequalit ies in Equation (8), w e get   K ( x i | ¯ x i − 1 ¯ y j ) −  K ( ¯ x i ) − K ( ¯ x i − 1 )    < O ( i + j ) . (9) W e next ev aluate K ( x i | ¯ x i − 1 ). F rom Lemma 4.4,   K ( x i | ¯ x i − 1 ) −  K ( x i ¯ x i − 1 ) − K ( ¯ x i − 1 )    < O ( i + j ) . (10) Using the inequalit y   K ( x i ¯ x i − 1 ) − K ( ¯ x i − 1 x i )   < O (1), we obtain   K ( x i | ¯ x i − 1 ) −  K ( ¯ x i ) − K ( ¯ x i − 1 )    < O ( i + j ) . (11) F rom Equations (9) and (11), the conclusion follo ws. Lemma 4.7 F or al l i , K ( x i y i | ¯ x i − 1 ¯ y i − 1 ) ≥ K ( x i | ¯ x i − 1 ¯ y i − 1 ) + K ( y i | ¯ x i − 1 ¯ y i − 1 ) − O ( i ) . Pro of . Th e conditional v ers ion of the inequalit y in Th eorem 4.3 holds tru e, i.e., for all strin gs u, v and w , K ( uv | w ) ≥ K ( u | w ) + K ( v | uw ) − O (log K ( u ) + log K ( v )). Thus, k eeping into accoun t that K ( x i ) ≤ | x i | + O (1) = 2 O ( i ) and K ( y i ) ≤ | y i | + O (1) = 2 O ( i ) , w e get K ( x i y i | ¯ x i − 1 ¯ y i − 1 ) ≥ K ( x i | ¯ x i − 1 ¯ y i − 1 ) + K ( y i | x i ¯ x i − 1 ¯ y i − 1 ) − O ( i ) . Note th at K ( y i | x i ¯ x i − 1 ¯ y i − 1 ) and K ( y i | ¯ x i ¯ y i − 1 ) are within a constan t of eac h other, and therefore K ( x i y i | ¯ x i − 1 ¯ y i − 1 ) ≥ K ( x i | ¯ x i − 1 ¯ y i − 1 ) + K ( y i | ¯ x i ¯ y i − 1 ) − O ( i ) . Next, we note that K ( y i | ¯ x i ¯ y i − 1 ) ≥ K ( y i | ¯ y i − 1 ) − O ( i ) ≥ K ( y i | ¯ x i − 1 ¯ y i − 1 ) − O ( i ), where the first inequalit y is deriv ed from Lemma 4.6. The co nclusion follo ws. 7 4.3 Regular functions The construction of z from x and y pro ceeds blo ck- wise: we take as in puts the blo c ks x i and y i and, from them, we build z i , the i -th blo c k of z . Th e input strings x i and y i , b oth of length n i , ha v e Kolmogoro v complexit y σ n i , for s ome p ositive constant σ , and the goal is to prod uce z i , of length m i (whic h will b e sp ecified later), with K olmogoro v co mplexit y (1 − ǫ ) m i , f or p ositiv e ǫ arbitrarily small. This r esem bles the f unctionalit y of rand omness extractors and, indeed, the follo w ing d efi nition captur es a p rop erty similar to that of extractors th at is sufficient for our purp oses. Definition 4.8 A function f : { 0 , 1 } n × { 0 , 1 } n → { 0 , 1 } m is ( σ , c ) -r e gular, if for an y k 1 , k 2 ≥ σ n , any two sub sets B 1 ⊆ { 0 , 1 } n and B 2 ⊆ { 0 , 1 } n with k B 1 k = 2 k 1 and k B 2 k = 2 k 2 have the fol lowing pr op erty: for any a ∈ { 0 , 1 } m , k f − 1 ( a ) ∩ ( B 1 × B 2 ) k ≤ c 2 m k B 1 × B 2 k . Remarks: Let [ N ] b e the set { 1 , . . . , N } . W e identify in the standard w a y { 0 , 1 } n with [ N ], where N = 2 n . W e can view [ N ] × [ N ] as a table with N rows and N columns and a function f : [ N ] × [ N ] → [ M ] as an assignment of a color c hosen from [ M ] to ea c h cell of the table. The fu nction f is ( σ , c )-regular if in any rectangle of size [ K ] × [ K ], with k ≥ σn , no color app ears more than a fraction of c/ M times. (The notion of regularit y is in teresting for small v alues of c b ecause it sa ys that in all r ectangles, unless they are sm all, all the colors app ear appro ximately th e same n umb er of times; note th at if c = 1, then all the colors ap p ear the same n umber of times.) W e sh o w using the p robabilistic metho d that f or an y σ > 0, ( σ , 2)-regular functions exist. Since the r egularit y p rop erty for a function f (give n via its truth table) can b e effectiv ely tested, w e can effect ive ly construct ( σ, 2)- regular functions b y exhaustiv e searc h W e tak e f : [ N ] × [ N ] → [ M ], a random function. First we sho w that with p ositiv e probabilit y such a fu n ction satisfies the defin ition of r egularit y for sets A and B ha vin g size 2 k , where k is exactly ⌈ σn ⌉ . Let’s te mp orarily call this prop ert y the w e ak r e gularity prop ert y . W e w ill sho w that in fact weak r egularit y imp lies the regularity prop ert y as defined ab ov e (i.e., the r egularit y should h old for all sets B 1 and B 2 of size 2 k 1 and resp ectiv ely 2 k 2 , for k 1 and k 2 gr e ater or e qu al ⌈ σn ⌉ ). Lemma 4.9 F or every σ > 0 , if M ≤ N 0 . 99 σ , then it holds with pr ob ability > 0 that f satisfies the ( σ, 2) - we ak r e gularity pr op erty as define d ab ove. Pro of . . Fix B 1 ⊆ [ N ] with k B 1 k = N σ (to k eep the notatio n simple, w e ignore truncation issues). Fix B 2 ⊆ [ N ] with k B 2 k = N σ . Let j 1 ∈ B 1 × B 2 and j 2 ∈ [ M ] b e fixed v alues. As d iscussed ab ov e, w e view [ N ] × [ N ] as a table with N rows and N columns. Then B 1 × B 2 is a rectangle in the table, j 1 is a cell in the rectangle, and j 2 is a co lor out of M p ossible colors. Clearly , Prob( f ( j 1 ) = j 2 ) = 1 / M . By Chernoff b ounds, Prob  no. of j 2 -colored cells in B 1 × B 2 N σ · N σ − 1 M  > 1 M  < e − (1 / M ) · N σ · N σ · (1 / 3) . By the union bou n d Prob( the a b o v e holds for some j 2 in [ M ] ) < M e − (1 / M ) · N σ · N σ · (1 / 3) . (12) 8 The n umber of recta ngles B 1 × B 2 is  N N σ  ·  N N σ  ≤   eN N σ  N σ  2 = e 2 N σ · e 2 N σ · (1 − σ ) ln N . (13) Note that if th ere is no rectangle B 1 × B 2 and j 2 as ab ov e, then f satisfies the w eak er ( σ , 2)-regularit y prop ert y . Therefore w e need that the pro duct of the righ t hand sides in equatio ns (12 ) and (13 ) is < 1. This is equiv alen t to (1 / M ) · N 2 σ · 1 / 3 − ln( M ) > 2 N σ + 2 N σ · (1 − σ ) ln N , whic h holds true for M ≤ N 0 . 99 σ . As promised, w e sho w next that w eak regularit y imp lies regularit y . Lemma 4.10 L e t f : { 0 , 1 } n × { 0 , 1 } n → { 0 , 1 } m such that for every B 1 ⊆ { 0 , 1 } n with k B 1 k = 2 k , for every B 2 ⊆ { 0 , 1 } n with k B 2 k = 2 k , and for e very a ∈ { 0 , 1 } m it holds that k f − 1 ( a ) ∩ ( B 1 × B 2 ) k ≤ p. Then for every k 1 ≥ k and every k 2 ≥ k , for every B ′ 1 ⊆ { 0 , 1 } n with k B ′ 1 k = 2 k 1 , for eve ry B ′ 2 ⊆ { 0 , 1 } n with k B ′ 2 k = 2 k 2 , and for e very a ∈ { 0 , 1 } m it holds that k f − 1 ( a ) ∩ ( B ′ 1 × B ′ 2 ) k ≤ p. Pro of . . W e p artition B ′ 1 and B ′ 2 in to subsets of size 2 k . So, B ′ 1 = A 1 ∪ A 2 ∪ . . . ∪ A s , with k A i k = 2 k , i = 1 , . . . , s and B 2 = C 1 ∪ C 2 ∪ . . . ∪ C t , with k C j k = 2 k , j = 1 , . . . , t . T h en, k f − 1 ( a ) ∩ ( B ′ 1 × B ′ 2 ) k = P s i =1 P t j =1 k f − 1 ( a ) ∩ ( A i × C j ) k ≤ P s i =1 P t j =1 p · k A i × C j k = p · k B ′ 1 × B ′ 2 k . 4.4 Increasing the randomness rate W e pro ceed to the proof of our main result, Theorem 1.1. W e give a “global” description of the effectiv e mappin g f : Q × { 0 , 1 } ∞ × { 0 , 1 } ∞ → { 0 , 1 } ∞ . It w ill b e clear ho w to obtain the n -th bit of the output in fin itely many steps, as it is formally required. Construction Input: τ ∈ Q ∩ (0 , 1], x, y ∈ { 0 , 1 } ∞ (the s equ ences x and y are oracles to which the pro cedur e has acc ess). Step 1: Split x in to x 1 , x 2 , . . . , x i , . . . and split y into y 1 , y 2 , . . . , y i , . . . , as describ ed in Section 4.2 ta king σ = τ / 2 and σ ′ = τ / 4. F or eac h i , let | x i | = | y i | = n i (as describ ed in Secti on 4.2). By Lemma 4. 2, K ( x i | ¯ x i − 1 ) > σ n i and K x ( y i | ¯ y i − 1 ) > σ n i . Step 2: As discussed in Section 4.3, for eac h i , constru ct by exhau s tiv e searc h E i : { 0 , 1 } n i × { 0 , 1 } n i → { 0 , 1 } m i a ( σ / 2 , 2)-regular function, where m i = i 2 . W e recall that this means that f or all k 1 , k 2 ≥ ( σ / 2) n i , for all B 1 ⊆ { 0 , 1 } n i with k A k ≥ 2 k 1 , for all B 2 ⊆ { 0 , 1 } n i with k B 2 k ≥ 2 k 2 , and for al l a ∈ { 0 , 1 } m i , k E − 1 i ( a ) ∩ B 1 × B 2 k ≤ 2 2 m i k B 1 × B 2 k . W e tak e z i = E i ( x i , y i ). Finally z = z 1 z 2 . . . z i . . . . 9 It is obvious that the ab o v e p ro cedure is a tru th-table reduction (i.e., it halts on all inputs). In w h at follo ws we will assume that the tw o in put sequences x and y ha ve r andomness rate τ and our goal is to show that the output z h as randomness rat e (1 − δ ) for an y δ > 0. Lemma 4.11 F or any ǫ > 0 , for al l i sufficiently lar ge, K ( z i | ¯ x i − 1 ¯ y i − 1 ) ≥ (1 − ǫ ) · m i . Pro of . Supp ose K ( z i | ¯ x i − 1 ¯ y i − 1 ) < (1 − ǫ ) · m i . Let A = { z ∈ { 0 , 1 } m i | K ( z | ¯ x i − 1 ¯ y i − 1 ) < (1 − ǫ ) · m i } . W e hav e k A k < 2 (1 − ǫ ) m i . Let t 1 , t 2 , B 1 , B 2 b e defined as f ollo w s : • t 1 = K ( x i | ¯ x i − 1 ¯ y i − 1 ). Since K ( x i | ¯ x i − 1 ) > σ n i , and taking in to accoun t Lemma 4.6, it follo ws that t 1 > σ n i − O ( i ) > ( σ / 2) n i , for all i su fficien tly large. • t 2 = K ( y i | ¯ x i − 1 ¯ y i − 1 ). By the same argu m en t as ab o v e, t 2 > ( σ / 2) n i . • B 1 = { x ∈ { 0 , 1 } n i | K ( x | ¯ x i − 1 ¯ y i − 1 ) ≤ t 1 } . • B 2 = { y ∈ { 0 , 1 } n i | K ( y | ¯ x i − 1 ¯ y i − 1 ) ≤ t 2 } . W e ha ve k B 1 k ≤ 2 t 1 +1 . T ak e B ′ 1 suc h that k B ′ 1 k = 2 t 1 +1 and B 1 ⊆ B ′ 1 . W e ha ve k B 2 k ≤ 2 t 2 +1 . T ak e B ′ 2 suc h that k B ′ 2 k = 2 t 2 +1 and B 2 ⊆ B ′ 2 . The b ounds on t 1 and t 2 imply th at B ′ 1 and B ′ 2 are large enough for E i to satisfy the regularit y prop ert y on them. In other w ords, for a ny a ∈ { 0 , 1 } m i , k E − 1 i ( a ) ∩ B ′ 1 × B ′ 2 k ≤ 2 2 m i k B ′ 1 × B ′ 2 k . So, k E − 1 i ( A ) ∩ B 1 × B 2 k ≤ k E − 1 i ( A ) ∩ B ′ 1 × B ′ 2 k = P a ∈ A k E − 1 i ( a ) ∩ B ′ 1 × B ′ 2 k ≤ 2 (1 − ǫ ) m i 2 2 m i k B ′ 1 × B ′ 2 k ≤ 2 t 1 + t 2 − ǫm i +3 . There is an algorithm that, give n ( x 1 , x 2 , . . . , x i − 1 ), ( y 1 , y 2 , . . . , y i − 1 ), (1 − ǫ ) m i , t 1 and t 2 , en ters an infi nite lo op during whic h it en umerates the elemen ts of the set E − 1 i ( A ) ∩ B 1 × B 2 . Therefore, the Kolmogoro v complexit y of an y elemen t of E − 1 i ( A ) ∩ B 1 × B 2 is b ounded by its rank in some fixed enumeration of th is set, the binary encodin g of the in put (including the information needed to separate the differen t comp onen ts), plus a constan t num b er of bits describing the en umeration pro cedur e. F ormally , f or ev ery ( u, v ) ∈ E − 1 i ( A ) ∩ B 1 × B 2 , K ( uv | ¯ x i − 1 ¯ y i − 1 ) ≤ t 1 + t 2 − ǫm i + 2(log(1 − ǫ ) m i + log t 1 + log t 2 ) + O (1) = t 1 + t 2 − Ω( i 2 ) . W e to ok in to accoun t that m i = i 2 , log t 1 = O ( i ), and log t 2 = O ( i ). In particular, K ( x i y i | ¯ x i − 1 ¯ y i − 1 ) ≤ t 1 + t 2 − Ω( i 2 ) . On the other hand, by L emm a 4.7, K ( x i y i | ¯ x i − 1 ¯ y i − 1 ) ≥ K ( x i | ¯ x i − 1 ¯ y i − 1 ) + K ( y i | ¯ x i − 1 ¯ y i − 1 ) − O ( i ) = t 1 + t 2 − O ( i ) . The last t wo inequations are in conflict, and thus we ha ve reac hed a con tradiction. The follo win g lemma concludes the pro of of the main result. 10 Lemma 4.12 F or any δ > 0 , the se quenc e z obtaine d by c onc atenating in o r der z 1 , z 2 , . . . , has r andomness r ate at le ast 1 − δ . Pro of . T ak e ǫ = δ / 4. By Lemma 4. 11 , K ( z i | ¯ x i − 1 ¯ y i − 1 ) ≥ (1 − ǫ ) · m i , for all i su fficien tly large. This implies K ( z i | z 1 . . . z i − 1 ) > (1 − ǫ ) m i − O (1) > (1 − 2 ǫ ) m i (b ecause eac h z j can b e effectiv ely computed from x j and y j ). By indu ction, it can b e sho wn th at K ( z 1 . . . z i ) ≥ (1 − 3 ǫ )( m 1 + . . . + m i ). F or the in ductiv e step, w e ha v e K ( z 1 z 2 . . . z i ) ≥ K ( z 1 . . . z i − 1 ) + K ( z i | z 1 . . . z i − 1 ) − O (log ( m 1 + . . . + m i − 1 ) + log( m i )) ≥ (1 − 3 ǫ )( m 1 + . . . + m i − 1 ) + (1 − 2 ǫ ) m i − O (log( m 1 + . . . + m i )) > (1 − 3 ǫ )( m 1 + . . . + m i ) . No w consider some z ′ whic h is b et w een z 1 . . . z i − 1 and z 1 . . . z i , i.e., for some strings u and v , z ′ = z 1 . . . z i − 1 u and z 1 . . . z i = z ′ v . S upp ose K ( z ′ ) < (1 − 4 ǫ ) | z ′ | . Then z 1 . . . z i − 1 can b e reco nstr u cted from: (a) the descriptor o f z ′ , whic h tak es (1 − 4 ǫ ) | z ′ | ≤ (1 − 4 ǫ )( m 1 + . . . + m i ) bits, (b) the string u w hic h tak es | z ′ | − | z 1 . . . z i − 1 | ≤ m i bits (c) O (log m i ) bits needed for separating u from the descriptor of z ′ and for describing the reconstruction pro cedur e. This implies that K ( z 1 . . . z i − 1 ) ≤ (1 − 4 ǫ )( m 1 + . . . + m i ) + m i + O (log m i ) = (1 − 4 ǫ )( m 1 + . . . + m i − 1 ) + (2 − 4 ǫ ) m i + O (log m i ) =  1 − 4 ǫ + (2 − 4 ǫ ) m i m 1 + ... + m i − 1  · ( m 1 + . . . + m i − 1 ) + O (log m i ) < (1 − 3 ǫ )( m 1 + . . . + m i − 1 ) . (The last inequalit y h olds if m i m 1 + ... + m i − 1 go es to 0, whic h is tru e for m i = i 2 .) This is a con tradiction. Th us w e ha v e p ro v ed that for ev ery n sufficien tly large, K ( z (1 : n )) > (1 − δ ) n . The main result can b e stated in terms of constructiv e Hausd orff dimension, a notion in tro du ced in measure theory . The constructiv e Hausdorff dimension of a sequence x ∈ { 0 , 1 } ∞ turns out to be equal to lim inf K ( x (1: n )) n (see [Ma y02 , Ry a84, Sta05]). Corollary 4.13 F or any τ > 0 , ther e is a truth-table r e duction f such that if x ∈ { 0 , 1 } ∞ and y ∈ { 0 , 1 } ∞ ar e indep endent and have c onstructive Hausdorff dimension at le ast τ , then f ( x, y ) has Hausdorff dimension 1 . Mor e over, f is u ni f orm in the p ar ameter τ . W e n ext observ e that Theorem 1.1 can b e str en gthened b y r elaxing the r equiremen t r egarding the indep endence of the tw o input sequences. F or a fu nction g : N → R + , w e sa y that t w o sequences x ∈ { 0 , 1 } ∞ and y ∈ { 0 , 1 } ∞ ha v e d ep end en cy g , if for all natural n umb ers n and m , K ( x (1 : n )) + K ( y (1 : m )) − K ( x ( 1 : n ) y (1 : m )) ≤ O ( g ( n ) + g ( m )) . In Theorem 1.1, the assu mption is that the t wo inp ut s equences hav e d ep enden cy g ( n ) = log n . Using essentia lly the same p ro of as the one that demonstrated Theorem 1.1, one can obtain the follo w ing result. Theorem 4.14 F or any τ > 0 , ther e e xist 0 < α < 1 and a truth-table r e duction f : { 0 , 1 } ∞ × { 0 , 1 } ∞ → { 0 , 1 } ∞ such that if x ∈ { 0 , 1 } ∞ and y ∈ { 0 , 1 } ∞ have dep endency n α and r andomness r ate τ , then f ( x, y ) has r andomness r ate 1 − δ , for any p ositive δ . Mor e over, f is uniform in the p ar ameter τ . 11 In T h eorem 1.1 it is required that the initial segments of x and y hav e K olmogoro v co mplexit y at least τ · n , f or a p ositiv e constan t τ . W e do not kno w if it is p ossible to obtain a s imilar resu lt for sequ ences with low er Kolmog oro v co mplexity . Ho wev er, using the same pr o of technique, it can b e sho wn that if x and y h av e their initial segmen ts with K olmogoro v complexit y only Ω(log n ), then one can p ro duce an infinite sequence z that has very high Kolmogo rov complexit y for infinitely man y of its prefixes. Theorem 4.15 F or any δ > 0 , ther e exist a c onstant C and a truth-table r e duction f : { 0 , 1 } ∞ × { 0 , 1 } ∞ → { 0 , 1 } ∞ with the fol lowing pr op erty: If the input se quenc es x and y ar e indep endent and satisfy K ( x (1 : n )) > C · log n and K ( y (1 : n )) > C · log n , for every n , then the output z = f ( x, y ) satisfies K ( z (1 : n )) > (1 − δ ) · n , for infinitely many n . F urthermor e, ther e is an infinite c omputable set S , such that K ( z (1 : n )) > (1 − δ ) · n , for ev e ry n ∈ S . 5 Ac kno wledgmen ts The author thanks T ed Slaman for b ringing to his atten tion the p r oblem of extracting Kol- mogoro v complexit y from in finite sequences, during the Dagstuhl seminar on Kolmogoro v complexit y in Jan uary 2006. The author is grateful to Cr istian Calud e for insight ful discus- sions. References [BDS07] L. Bienv enu, D. Dot y , and F. S tephan. Cons tructiv e d imension and wea k truth-table degrees. In Computation and L o gic in the R e al W orld - Thir d Confer enc e of Com- putability in Eur op e . Springer-V erlag L e ctur e Notes in Computer Scienc e #44 97 , 2007. T o App ear. Av ailable as T echnical Rep ort arXiv:c s/0701089 ar [BFNV05 ] H. Buhrman, L. F ortno w, I. Newman, and N. V ereshchagin. Increasing Kolmogoro v complexit y . In Pr o c e e dings of the 22nd Annual Symp osium on The or etic al Asp e cts of Comp uter Scienc e , pages 412–42 1, Berlin, 2005. S pringer-V erlag L e ctur e Notes in Computer Scienc e #3404 . [BIW04] B. Barak, R. Imp agliazz o, and A. Wigderson. Extracting randomness using few indep end en t sources. In Pr o c e e dings of the 36th ACM Symp osium on The ory of Computing , pages 384– 393, 2004. [Dot07 ] D. Dot y . Dimension extractors and optimal decompression. T ec hnical Rep ort arXiv:cs/060 6078 , Computing Researc h Rep ository , arXiv.org, Ma y 2007. T o ap- p ear in Theory of Computing Sys tems. [FHP + 06] L. F ortno w, J. Hitc h co c k, A. P a v an, N.V. Vin o dc handr an, and F. W ang. Extract- ing K olmogoro v complexity w ith applications to dimension zero-one la ws. I n Pr o- c e e dings of the 33r d Internatio nal Col lo quium on Automata, L anguages, and Pr o- gr amming , pages 335–345, Berlin, 2006. S pringer-V erlag L e ctur e Notes in Computer Scienc e #4051 . [Lut03] J. Lutz. The dimensions of individu al strings and sequences. Informa tion and Contr ol , 187:49–7 9, 2003. [Ma y02] E. Ma y ordomo. A Kolmogoro v complexity charact erization of constructive Haus- dorff dimension. Information Pr o c essing L etters , 84: 1–3, 2002. 12 [MN06] J. Miller and A. Nies. Randomness and compu tability . Op en qu estions. Bul l. Symb. L o gic , 12(3):3 90–410, 2006. [NR06] A. Nies and J. Reimann. A lo we r cone in the wtt d egrees of non-inte gral effectiv e dimension. In Pr o c e e dings of IM S workshop on Computational Pr osp e cts of Infinity , Singap ore, 2006. T o app ear. [Rei04] J. Reimann. Compu tabilit y and fractal dimension. T ec hn ical r ep ort, Univ ersit¨ at Heidelb erg, 2004. Ph.D. thesis. [Ry a84] B. Ryabk o. C o ding of com b inatorial sour ces and Hausdorff dimension. Dokla dy Akad emii Nauk SSR , 277 :1066–10 70, 1984. [She00] A. S hen. Algorithmic information theory and Kolmog oro v complexit y . T ec hnical Rep ort 2000- 034, Uppsala Univ ersitet, Decem b er 2000. [Sta05] L. S taiger. Con s tructiv e dimension equals Kolmogoro v complexit y . In forma- tion Pr o c essing L etters , 93:149– 153, 2005. Preliminary version: Researc h Rep ort CDMTCS-210, Univ. of Auc kland , Jan uary 2003. 13