A Note on Topology Preservation in Classification, and the Construction of a Universal Neuron Grid

A Note on T opology Preserv ation in Classification, and the Construction of a Uni v ersal Neuron Grid Dietmar V olz dietmar.volz@gmx.net Nov ember 2, 2021 Abstract It will be shown that according to theorems of K. Menger, ev ery neuron grid if identified with a curve is able to preserv e the adopted qualitative structure of a data space. Furthermore, if this identification is made, the neuron grid structure can always be mapped to a subset of a uni versal neuron grid which is constructable in three space dimensions. Conclusions will be drawn for established neuron grid types as well as neural fields. 1 Mathematical Pr eliminaries T opology is one of the basic branches of mathematics. It is sometimes also referred to as qualitativ e geometry , in a way that it deals with the qualitative properties and struc- ture of geometrical objects. The geometrical objects of interest in this paper are vector spaces, manifolds, and curves. These form the basis of the presented mathematical treatment of clustering with neuron grids. Consequently , the paper has to begin with some mathematical preliminaries. 1.1 Manif olds In the follo wing, a n-dimensional v ector space will be identified with a subspace of I R n . It is assumed that I R n is equipped with a topology which in turn is induced by a metric. Mappings between subspaces of I R n are called homeomorphic or topology preserving if they are one-to-one and continuous in both directions. It is kno wn from the theorem of dimension in variance (Brouwer, 1911) that mappings between non-empty open sets U ⊂ I R m and V ⊂ I R n for m 6 = n are ne ver homeomorphic. In the light of Brouwer’ s theorem it is the open sets that fix the topological dimension of a subset of I R n . As the I R n is introduced as a metric space, the open sets are gi ven by open balls that formalize the concept of distance between points of this metric space. Homeomorphic mappings preserve the neighborhood relationship between points of I R n . Amongst the huge v a- riety of subsets of the metric space I R n , the n-dimensional manifolds (or n-manifolds) hav e turned out to be of interest as these describe solution spaces of equations or ge- ometrical entities. Manifolds are parameterized geometrical objects, parameters could describe e.g. the coordinates of a physical space. As n-manifolds are locally Euclidean of dimension n they are subject to Brouwer’ s theorem and therefore of fix ed topological 1 dimension. As a consequence, dimension reducing mappings between manifolds will not be able to transfer mutual topological structures. A vi vid e xample of the dimension conflict of two manifolds of different dimension is given by a surjecti ve and continu- ous mapping of [ 0 , 1 ] onto [ 0 , 1 ] × [ 0 , 1 ] which is also called a ’Peano curve’ (Peano, 1890). The con voluted structure of a Peano curve does not reflect the neighborhood relationship of elements of the underlying manifold ⊂ I R 2 , not e ven locally: There is no topology preserving mapping of [ 0 , 1 ] onto [ 0 , 1 ] × [ 0 , 1 ] . An illustration of a Peano curve to 4 t h iteration is giv en in figure 1. Figure 1: A Peano curve (source: W ikipedia) 1.2 Curves The Peano curve introduced in the preceding section is based on the con ventional def- inition of a compact curve as a continuous mapping of [ 0 , 1 ] . The following definition according to (Menger, 1968) renders the definition of a compact curve more precisely . Beforehand, a definition of a continuum is required. Definition A compact, connected set ⊂ I R n having more than one element is a contin- uum. A set ⊂ I R n which contains no continuum is called discontinuous . Definition A continuum K as a subset of a metric space is called a curve if every point of K is contained in arbitrary small neighborhoods having discontinuous intersects with K . From Menger’ s definition of a curve the following theorem results: Theorem [Menger] Every curve defined in a metric space is homeomorphic to a curve defined in I R 3 . The proof of this theorem is left here. The interested reader is referred to Menger’ s textbook (Menger, 1968). The theorem states that with regard to topological aspects, the transition from curv es defined in three-dimensional Euclidean space to curves de- 2 fined in an arbitrary metric space doesn’t gi ve an y generalization. Menger’ s definition of a curve ev en leads to another theorem (Menger, 1968). Theorem [Menger] Every compact curve in a metric space can be mapped to a subset of a so-called universal curve ⊂ I R 3 . The universal curve K is constructable and up to homeomorphism uniquely defined. The construction starts with a cube [ 0 , 1 ] 3 ⊂ I R 3 . K is the set of all points of [ 0 , 1 ] 3 such that at least two of the three coordinates have triadic e xpansions that do not contain a 1. The hereby constructed set fulfills the definition of a curve (Menger, 1968) and leads to a self-similar structure. The construction is sho wn in figure 2. Figure 2: Menger’ s universal curv e in perspectiv e view Summarizing, the topological relationships that arise from Brouwer’ s and Menger’ s theorems are as follows: X ⊂ I R n 6 ∼ = − − − − → Y ⊂ I R m 6 ∼ = − − − − → Z ⊂ I R 3 , if m 6 = n and m 6 = 3 C ⊂ I R n ∼ = − − − − − − − − − − − − − − − − − − − − − → C ⊂ I R 3 , ∀ n hereby X, Y , Z are (n,m,3)-manifolds, C a curve, ∼ = identifies a homeomorphic mapping. Subsets of the universal curve as introduced in this section usually are no manifolds. The only exception are the trivial cases of subsets that are homeomorphic to an interval or S 1 . 3 2 A pplication to Neuron Grids 2.1 V ector Quantization The method of V ector Quantization (VQ) (Linde et al., 1980) classifies elements of a vector space V ⊂ I R n by approximating the probability density function p ( v ) of el- ements v of V . Hereby , the elements of V are mapped iterati vely to a set of weight vectors U such that respecti ve a suitable metric the quantization error functional gets minimal. An extension of the VQ method introduces a topology on the set of weight vectors such that U constitutes a topological space which is then called a neur on grid . By means of the topology of the neuron grid, elements of U are mutually adjacent if they are adjacent on the neuron grid. Using algorithms of machine learning, for e x- ample Kohonen’ s feature maps (Kohonen, 1990) and several variants thereof (Fritzke, 1992; Oja et al., 1999) try to transfer the topology of the v ector space V to the topology of the neuron grid U . Hereby , the topology of the neuron grid U as adapted to the vector space topology V is representative of the topology of V . The following section will provide some insight to the restrictions of topology preservation in clustering of a vector space with VQ methods. 2.2 Identification of a Neuron Grid with a Cur ve Assuming Menger’ s curve definition, the following identification is made: Definition A neur on grid is a curve It is required that neurons of a neuron grid constitute a discrete point set of a curve. Subsets of a neuron grid which contain no neurons are called links between neurons. Consequently , the topology of the defined neuron grid corresponds to a curve topology . The identification of a neuron grid with a curv e as moti vated in this section is coherent and straightforw ard for neuron grids whose topologies are identifiable with a subset of Menger’ s uni versal curve. This is the case for neuron grids of the Kohonen type (K ohonen, 1990) as well as some variants thereof e.g. (Fritzke, 1992; Oja et al., 1999) that constitute topological spaces of a discrete point set together with a set of links between the point set. The presented identification with a curve produces the following results: i All neuron grid topologies are uniquely definable in I R 3 . There’ s no need to intro- duce ’hypercubes’ or ’high-dimensional’ grid topologies. ii Giv en the neuron grid topology is adapted to a vector space topology ⊂ I R n , then the neuron grid topology will generally be retained in I R 3 . Clustering results are always visualizable in I R 3 . iii Up to homeomorphism it e xists a uniquely defined universal neur on grid such that ev ery connected neuron grid is a subset of the univ ersal neuron grid. These results were inferred directly from Menger’ s curve theorems. It should be mentioned though that Brouwer’ s theorem precludes a direct topology pre- serving mapping between the vector space topology V which is induced by the metric of I R n and U ⊂ I R m if m 6 = n . The conflict of dimension as illustrated in the example of section 1.1 is hereby reproduced, this time as the lack of homeomorphic mappings be- tween open balls of metric spaces. In consequence, metrical properties of I R n generally won’ t be reproduced on the clustering result. 4 It should also be mentioned that similar results are not possible for a neuron grid if it is defined by a two- or higher-dimensional manifold. The well-kno wn ’Klein bottle’ (Klein, 1923) provides a vi vid example what might happen to a two-dimensional sur- face that is defined in I R n , n > 3, and shows an apparent self-intersection if mapped to I R 3 . The self-intersection generates a subset of points on the Klein bottle that are sep- arate in I R n , n > 3 and get identified through the mapping to I R 3 . Hereby , two separate clusters are spuriously merged into one. Discussion The presented theorems of Brouwer and Menger illustrate an alternati ve though sim- ple and straightforward approach to the topological foundation of neuron grids. The identification of a neuron grid with a subset of the univ ersal curve at a first or sec- ond glance may be seen as tri vial. This is in particular due to the presented definition of a curve which is intuiti ve and comprehensible, also from a nai ve standpoint. The presented topological framework of neuron grids restricts neuron sets to discrete point sets. Alternati ve approaches (Amari, 1977; Bresslof f, 2005; Cottet, 1995) introduce continuous models of neural fields. As neural fields per definition provide open sets in their domain, the present paper also incloses the ca veat that neural fields might produce unusable results if applied to clustering tasks. Acknowledgements I would like to thank Catalin Dartu for the nice visualization of Menger’ s uni versal curve. Refer ences Amari, S. (1977). Dynamics of pattern formation in lateral-inhibition type neural fields. Biol Cybern , 27 , 77 – 87. Bressloff, P . C. (2005). Spontaneous symmetry breaking in self-organizing neural fields. Biol Cybern , 93(4) , 256 – 274. Brouwer , L. E. J. (1911). Beweis der Inv arianz der Dimensionszahl. Math. Ann. , 70 , 161 – 165 Cottet, G. (1995). Neural networks: Continuous approach and applications to image processing. J ournal of Biological Systems , 3 , 1131 – 1139. Fritzke, B. (1992). Growing Cell Structures - A Self-Organizing Network in k Di- mensions. Artificial Neur al Networks. North-Holland. Amster dam , V ol. II , 1051 – 1056. Klein, F . (1923). Gesammelte Mathematische Abhandlungen. Spring er . Berlin , Bd.III , 571 5 K ohonen, T . (1990). The self-organizing map. Pr oceedings of the IEEE , 78(9) , 1464 – 1480. Linde, Y ., Buzo, A., & Gray , R. M. (1980). An algorithm for vector quantizer design. IEEE T rans. Comm. , 28 , 84 – 95. Menger , K. (1968). Kurv entheorie. Chelsea Pub Co , 347 – 360. Oja, E., Laaksonen, J., & K oskela, M. (1999). Application of T ree Structured Self- Organizing Maps in Content-Based Image Retriev al. Pr oc. of the 9th Int. Conf. on ANN (ICANN’99) , V ol. 1 , 174 – 179. Peano, G. (1890). Sur une courbe, qui remplit toute une aire plane. Math. Ann. , 36 , 157 – 160. 6