Capacity Achieving Codes From Randomness Condensers

We establish a general framework for construction of small ensembles of capacity achieving linear codes for a wide range of (not necessarily memoryless) discrete symmetric channels, and in particular, the binary erasure and symmetric channels. The main tool used in our constructions is the notion of randomness extractors and lossless condensers that are regarded as central tools in theoretical computer science. Same as random codes, the resulting ensembles preserve their capacity achieving properties under any change of basis. Using known explicit constructions of condensers, we obtain specific ensembles whose size is as small as polynomial in the block length. By applying our construction to Justesen’s concatenation scheme (Justesen, 1972) we obtain explicit capacity achieving codes for BEC (resp., BSC) with almost linear time encoding and almost linear time (resp., quadratic time) decoding and exponentially small error probability.

💡 Research Summary

The paper presents a unified framework for constructing small ensembles of capacity‑achieving linear codes for a broad class of discrete symmetric channels, with particular emphasis on the binary erasure channel (BEC) and binary symmetric channel (BSC). The central technical tool is the use of randomness extractors and lossless condensers—objects originally developed in theoretical computer science for pseudorandomness. By interpreting the seed of an extractor or condenser as a choice of inner code, the authors show that the parameters of these objects (seed length, error, output length) directly control the size of the code ensemble, the decoding error probability, and the gap to channel capacity.

The work revisits Justesen’s concatenation scheme, which combines an outer high‑rate MDS (or Reed–Solomon) code with a collection of short inner codes. Classical implementations required an exponentially large inner‑code ensemble to guarantee that a large fraction of the inner codes were capacity‑achieving. By replacing the traditional inner‑code construction with one derived from a lossless condenser, the authors achieve the same probabilistic guarantees with ensembles whose size is only polynomial in the block length. In particular, the Guruswami‑Umans‑Vadhan condenser, which has logarithmic seed length and negligible error, yields ensembles of size n^c for some constant c, while preserving the essential “random‑like” property that each non‑zero vector belongs to only a few codes in the ensemble.

For the BEC, the authors set the condenser’s output length to match the channel’s capacity 1‑p (minus an arbitrarily small loss δ). The resulting inner codes have rate 1‑p‑δ and, when concatenated with the outer MDS code, produce overall codes of rate arbitrarily close to capacity. The decoding error decays exponentially in the block length (≈exp(−Ω(n))) and both encoding and decoding run in almost linear time (O(n log n)).

For the BSC, a similar parameter choice yields inner codes of rate 1‑h(p)‑δ, where h(p) is the binary entropy function. The concatenated construction again achieves rates arbitrarily close to the Shannon capacity, with exponentially small error probability. Decoding can be performed by standard syndrome‑based algorithms; the overall decoding complexity is almost quadratic (O(n^2)) but still polynomial, while encoding remains near‑linear.

The paper also proves a duality theorem linking linear affine extractors and lossless condensers: the transpose of a linear extractor matrix serves as a condenser with complementary parameters. This structural insight may be useful beyond the immediate coding applications, for example in constructing pseudorandom objects for other computational tasks.

In summary, the authors demonstrate that by leveraging explicit extractor and condenser constructions, one can obtain explicit, efficiently encodable and decodable capacity‑achieving codes with ensembles of polynomial size. The resulting codes combine the theoretical optimality of random coding arguments with practical feasibility, narrowing the gap between provable constructions and the performance of modern codes such as LDPC and turbo codes. This work thus opens a promising avenue for further exploration of pseudorandomness tools in coding theory.