Hierarchical Selective Recruitment in Linear-Threshold Brain Networks, Part I: Single-Layer Dynamics and Selective Inhibition

Hierar chical Selectiv e Recruitment in Linear -Thr eshold Brain Networks Part I: Single-Lay er Dynamics and Selectiv e Inhibition Erfan Nozari Jorge Cort ´ es Abstract —Goal-driven selective attention (GDSA) refers to the brain’ s function of prioritizing the activity of a task- rele vant subset of its overall network to efﬁciently process rele vant information while inhibiting the effects of distractions. Despite decades of resear ch in neuroscience, a comprehensiv e understanding of GDSA is still lacking. W e pr opose a novel framework using concepts and tools from contr ol theory as well as insights and structures from neuroscience. Central to this framework is an information-pr ocessing hierarch y with two main components: selecti ve inhibition of task-irrelev ant activity and top-down recruitment of task-rele vant activity . W e analyze the internal dynamics of each layer of the hierarchy described as a network with linear -threshold dynamics and deriv e conditions on its structure to guarantee existence and uniqueness of equilibria, asymptotic stability , and boundedness of trajectories. W e also pro vide mechanisms that enforce selective inhibition using the biologically-inspired schemes of feedforward and feedback inhi- bition. Despite their differences, both lead to the same conclusion: the intrinsic dynamical properties of the (not-inhibited) task- rele vant subnetworks are the sole determiner of the dynamical properties that are achievable under selective inhibition. I . I N T RO D U C T I O N The human brain is constantly under the inﬂux of sensory inputs and is responsible for inte grating and interpreting them to generate appropriate decisions and actions. This inﬂux contains not only the pieces of information relev ant to the present task(s), but also a myriad of distractions. Goal-driven selectiv e attention (GDSA) refers to the active selective pro- cessing of a subset of information inﬂux while suppressing the ef fects of others, and is vital for the proper function of the brain. 1 Examples range from selectiv e audition in a crowded place to selectiv e vision in cluttered en vironments to selective taste/smell in food. As a result, a long standing question in neuroscience inv olves understanding the brain’ s comple x mechanisms underlying selectiv e attention [2]–[7]. A central element in addressing this question is the role played by the hierarchical organization of the brain [8]. Broadly , this organization places primary sensory and motor areas at the bottom and inte grative association areas (prefrontal cortex in particular) at the top. Accordingly , sensory infor- mation is processed while ﬂowing up the hierarchy , where decisions are e ventually made and transmitted back down the hierarchy to generate motor actions. 2 The top-down direction A preliminary version appeared as [1] at the American Control Conference. Erfan Nozari is with the Department of Electrical and Systems Engineering, Univ ersity of Pennsylvania, Philadelphia, P A 19104, enozari@seas.upenn.edu. Jorge Cort ´ es is with the Department of Mechanical and Aerospace Engineer- ing, UC San Diego, La Jolla, CA 92093, cortes@ucsd.edu. 1 Note the distinction with stimulus-driven selecti ve attention (the reacti ve shift of focus based on saliency of stimuli) which is not the focus here. 2 Note that the role of memory (being distrib uted across the brain) is implicit in this simpliﬁed stimulus-response description. Indeed, many sensory inputs only form memories (without motor response) or many motor actions result chieﬂy from memory (without sensory stimulation). The hierarchical aspect is nevertheless present. is also responsible for GDSA, where the higher-order areas differentially “modulate” the activity of the lower -level areas such that only rele v ant information is further processed. This phenomenon constitutes the basis for GDSA and has been the subject of extensiv e experimental research in neuroscience, see e.g., [4], [9]–[18]. Howe ver , a complete understanding of how , when (how quick), or where (within the hierarchy) it occurs is still lacking. In particular , the relationship between GDSA and the dynamics of the in volv ed neuronal networks is poorly understood. Our goal is to address this gap from a model-based perspecti ve, resorting to control-theoretic tools to explain various aspects of GDSA in terms of the synaptic network structure and the dynamics that emerge from it. In this work, we propose a theoretical frame work, termed hierar chical selective r ecruitment (HSR) , to explain the net- work dynamics underlying GDSA. This framework consists of a novel hierarchical model of brain organization (though composed of well-established sub-models at each layer), a set of analytical results regarding the multi-timescale dynamics of this model, and a careful translation between the properties of these dynamics and well known experimental observ ations about GDSA. The starting point in the dev elopment of HSR is the observation that different stimuli, in particular the task- relev ant and task-irrelev ant ones, are processed by different populations of neurons (see, e.g., [4], [5], [7], [11]–[14], [18]). W ith each neuronal population represented by a node in the ov erall neuronal network of networks and based on extensi ve experimental research (see belo w), HSR primarily relies on the selective inhibition of the task-irrelev ant nodes and the top- down recruitment of the task-rele v ant nodes of each layer by the layer immediately abo ve. This paper analyzes the dynamics of individual layers as well as the mechanisms for selective inhibition in a bilayer network. These results set the basis for the study of the mechanisms for top-down recruitment in multilayer networks in our accompanying work [19]. Literatur e Review: In this work we use dynamical networks with linear -threshold nonlinearities (the unbounded version also called rectiﬁed linear units, ReLU, in machine learning) to model the activity of neuronal populations. Linear-threshold models allo w for a unique combination between the tractability of linear systems and the dynamical versatility of nonlinear systems, and thus have been widely used in computational neuroscience. They were ﬁrst proposed as a model for the lateral eye of the horseshoe crab in [20] and their dynam- ical behavior has been studied at least as early as [21]. A detailed stability analysis of symmetric (undirected) linear- threshold networks has been carried out in continuous [22] and discrete [23] time: howe ver , this has limited relev ance for biological neuronal networks, which are fundamentally asymmetric (due to the presence of excitatory and inhibitory neurons). Regarding asymmetric networks, it was claimed (without proper proof) in [24] that the identity minus the matrix of synaptic connectivities being a P-matrix is necessary and sufﬁcient for the existence and uniqueness of equilibria (EUE), the negativ e of this matrix being totally Hurwitz is necessary and suf ﬁcient for local asymptotic stability , and the matrix of synaptic connectivities being absolutely Schur stable is sufﬁcient for global asymptotic stability . In addition to lacking proper proof, these results were limited to fully- inhibitory networks. The latter assertion was later proved rigorously in [25] for arbitrary networks, while we prove the ﬁrst two (except on certain sets of measure zero) here. Around the same time, [26] considered the more general class of monotonically non-decreasing acti vation functions and prov ed the sufﬁciency of identity minus the matrix of synaptic connectivities being a P-matrix for the uniqueness of equilibria (being only one of the four implications we prov e here) and the sufﬁcienc y of the same matrix being L yapunov diagonally stable for global asymptotic stability (which we relax here by allo wing for arbitrary quadratic L yapunov functions). This work was later generalized to discontinuous neural networks (though not applicable to our model here) in [27]. Also related is the work [28] showing the necessity and sufﬁcienc y of identity minus the matrix of synaptic connectivities being a P 0 -matrix for EUE of similar systems but with strictly monotonically increasing activ ation functions. The work [29] provides a comprehensi ve revie w of stability analysis of a range of continuous-time recurrent neural networks, including the linear-threshold model. L yapunov-based methods hav e also been used in a num- ber of later studies for discrete-time linear-threshold net- works [30]–[32], but the extension of these results to continuous-time dynamics is unclear . In fact, the use of L yapunov-based techniques in continuous-time networks has remained limited to planar dynamics [33] and restricti ve condi- tions for boundedness of trajectories [33], [34]. Recently , [35] presents interesting properties of competiti ve (i.e., fully in- hibitory) linear-threshold networks, particularly regarding the emergence of oscillations. Howe ver , the majority of neurons in biological neuronal networks are excitatory , making the im- plications of these results limited. Moreov er, all the preceding works are limited to networks with constant e xogenous inputs whereas time-varying inputs are essential for modeling inter- layer connections in HSR. A critical property of linear -threshold networks is that their nonlinearity , while enriching their behavior be yond that of linear systems, is piecewise linear . Accordingly , almost all the theoretical analysis of these networks builds upon the formulation of them as switched afﬁne systems. There exists a vast literature on the analysis of general switched linear/af ﬁne systems, see, e.g., [36]–[38]. Ne vertheless, we ha ve found that the conditions obtained by applying these results to linear- threshold dynamics are more conservati ve than the ones we obtain using direct analysis of the system dynamics. This is mainly due to the fact that such results, by the essence of their generality , are oblivious to the particular structure of linear- threshold dynamics that can be leveraged in direct analysis. Selectiv e inhibition has been the subject of e xtensive re- search in neuroscience. A number of early studies [4], [11], [12] provided e vidence for a mechanism of selectiv e visual at- tention based on a biased competition between the subnetwork of task-rele v ant nodes and the subnetwork of task-irrele v ant ones. In this model, nodes belonging to these subnetworks compete at each layer by mutually suppressing the activity of each other , and this competition is biased to wards task-relev ant nodes by the layer immediately above. Later studies [13], [14] further supported this theory using functional magnetic resonance imaging (fMRI) and sho wed [39], in particular , the suppression of activity of task-irrele vant nodes as a result of GDSA. This suppression of acti vity is further sho wn to occur in multiple layers along the hierarchy [40], grow with increasing attention [41], [42], and be in versely related to the power of the task-irrele vant nodes’ state trajectories in the alpha frequency band ( ∼ 8 - 14 Hz ) [16]. Statement of Contrib utions: The contributions are twofold. First, we analyze the internal dynamics of a single-layer linear-threshold network as a basis for our study of hier- archical structures. Our results here provide a comprehen- siv e characterization of the dynamical properties of linear- threshold networks. Speciﬁcally , we show that existence and uniqueness of equilibria, asymptotic stability , and boundedness of trajectories can be characterized using simple algebraic conditions on the network structure in terms of the class of P-matrices (matrices with positiv e principal minors), totally- Hurwitz matrices (those with Hurwitz principal submatrices, shown to be a sub-class of P-matrices), and Schur-stable matrices, respecti vely . In addition to forming the basis of HSR, these results solve some long-standing open problems in the characterization of linear-threshold networks [21], [24], [25], [33]–[35] and are of independent interest. Our analysis cov ers both the class of unbounded (a.k.a. ReLU) as well as bounded linear -threshold networks, where the latter is a piecewise-af ﬁne approximation of sigmoidal neural networks, for which limited analytical results are av ailable. Our second contribution pertains the problem of selectiv e inhibition in a bilayer network. Moti v ated by the mechanisms of inhibition in the brain, we study feedforward and feedback mechanisms. W e provide necessary and suf ﬁcient conditions on the network structure that guarantee selecti ve inhibition of task-irrelev ant nodes at the lower-le vel while simultaneously guaranteeing various dynamical properties of the resulting (partly inhibited, partly activ e) subnetwork, including existence and uniqueness of equilibria and asymptotic stability . Interestingly , under both mechanisms, these conditions require that the (not-inhibited) task-relev ant part of the lower -lev el subnetwork intrinsically satisﬁes the same desired dynamical properties. This is par- ticularly important for selective inhibition as asymptotic sta- bility underlies it. The results un veil the important role of task-relev ant nodes in constraining the dynamical properties achiev able under selective inhibition and have implications for the number and centrality of nodes that need to be inhib- ited for an unstable-in-isolation subnetwork to gain stability through selecti ve inhibition. For subnetworks that are not stable as a whole, these results provide conditions on the task-relev ant/irrelev ant partitioning of the nodes that allow for stabilization using inhibitory control. I I . P R E L I M I NA R I E S W e introduce notational con ventions and basic concepts on matrix analysis and modeling of biological neuronal networks. Notation Throughout the paper , we employ the follo wing notation. W e use R , R ≥ 0 , and R ≤ 0 to denote the set of reals, nonneg- ativ e reals, and nonpositive reals, respectiv ely . W e use bold- faced letters for vectors and matrices. 1 n , 0 n , ` n , 0 p × n , and I n stand for the n -vector of all ones, the n -v ector of all zeros, the n -vector of all ` ’ s, the p -by- n zero matrix, and the identity n -by- n matrix (we omit the subscripts when clear from the context). Giv en a vector x ∈ R n , x i and ( x ) i refer to its i th component. Giv en A ∈ R p × n , a ij refers to the ( i, j ) th entry . For block-partitioned x and A , x i and A ij refer to the i th block of x and ( i, j ) th block of A , respecti vely . In block representation of matrices, ? denotes arbitrary blocks whose v alue is immaterial to the discussion. For A ∈ R p × n , range ( A ) denotes the subspace of R p spanned by the columns of A . If x and y are vectors, x ≤ y denotes x i ≤ y i for all i . For symmetric P ∈ R n × n , P > 0 ( P < 0 ) denotes that P is positi ve (neg ati ve) deﬁnite. Giv en A ∈ R n × n , its element-wise absolute value, determinant, spectral radius, and induced 2 -norm are denoted by | A | , det ( A ) , ρ ( A ) , and k A k , respectively . Similarly , for x ∈ R n , k x k is its 2 -norm. Like wise, for two matrices A and B , diag ( A , B ) denotes the block-diagonal matrix with A and B on its diagonal. Giv en a subspace W of R n , W ⊥ denotes the orthogonal complement of W in R n . For x ∈ R and m ∈ R > 0 ∪ {∞} , [ x ] m 0 = min { max { x, 0 } , m } , which is the projection of x onto [0 , m ] . When x ∈ R n and m ∈ R n > 0 ∪ {∞} n , we similarly de- ﬁne [ x ] m 0 = [[ x 1 ] m 1 0 · · · [ x n ] m n 0 ] T . All measure-theoretic statements are meant in the Lebesgue sense. Matrix Analysis W e here deﬁne and characterize se veral matrix classes of interest that play a key role in the forthcoming discussion. Deﬁnition II.1. (Matrix classes). A matrix A ∈ R n × n is (i) absolutely Schur stable if ρ ( | A | ) < 1 ; (ii) totally L -stable , denoted A ∈ L , if ther e exists P = P T > 0 such that ( − I + A T Σ ) P + P ( − I + ΣA ) < 0 for Σ = diag ( σ ) and all σ ∈ { 0 , 1 } n ; (iii) totally Hurwitz , denoted A ∈ H , if all the principal submatrices of A are Hurwitz; (iv) a P-matrix , denoted A ∈ P , if all the principal minors of A ar e positive. In working with P-matrices, the principal pivot transform of a matrix plays an important role. Giv en A =  A 11 A 12 A 21 A 22  , with nonsingular A 22 , its principal piv ot transform is the matrix π ( A ) ,  A 11 − A 12 A − 1 22 A 21 A 12 A − 1 22 − A − 1 22 A 21 A − 1 22  . Note that π ( π ( A )) = A . The next result formalizes sev eral equiv alent characterizations of P-matrices. Lemma II.2. (Properties of P-matrices [43], [44]). A ∈ R n × n is a P-matrix if and only if any of the following holds: (i) A − 1 is a P-matrix; (ii) all r eal eigen values of all the principal submatrices of A ar e positive; (iii) for any x ∈ R n \ { 0 } ther e is k such that x k ( Ax ) k > 0 ; (iv) the principal pivot transform of A is a P-matrix. The matrix classes in Deﬁnition II.1 have important inclu- sion relationships, as shown next. Lemma II.3. (Inclusions among matrix classes). F or A , W ∈ R n × n , we have (i) ρ ( | W | ) < 1 ⇒ − I + W ∈ H ; (ii) k W k < 1 ⇒ W ∈ L ; (iii) W ∈ L ⇒ − I + W ∈ H ; (iv) A ∈ H ⇒ − A ∈ P . Pr oof: (i) . From [45, Fact 4.11.19], we hav e that ρ ( | W σ | ) < 1 for any principal submatrix W σ of W , which implies ρ ( W σ ) < 1 by [45, Fact 4.11.17], implying the result. (ii) It is straightforward to check that P = I n satisﬁes ( − I + W T Σ ) P + P ( − I + ΣW ) < 0 for all σ ∈ { 0 , 1 } n . (iii) Pick an arbitrary σ ∈ { 0 , 1 } n and let the permutation Π ∈ R n × n be such that ΠΣW Π T =  0 0 ˆ W 21 ˆ W 22  , where ˆ W 22 is the principal submatrix of W corresponding to σ . Then P ( − I + ΣW ) = PΠ T  − I 0 ˆ W 21 − I + ˆ W 22  Π = Π T  ΠPΠ T | {z } ˆ P  − I 0 ˆ W 21 − I + ˆ W 22   Π = Π T  ? ? ? ˆ P 22 ( − I + ˆ W 22 )  Π , where ˆ P =  ˆ P 11 ˆ P 12 ˆ P 21 ˆ P 22  = ˆ P T > 0 . Thus, by assumption, Π T  ? ? ? ( − I + ˆ W T 22 ) ˆ P 22 + ˆ P 22 ( − I + ˆ W 22 )  Π < 0 ⇒  ? ? ? ( − I + ˆ W T 22 ) ˆ P 22 + ˆ P 22 ( − I + ˆ W 22 )  < 0 ⇒ ( − I + ˆ W T 22 ) ˆ P 22 + ˆ P 22 ( − I + ˆ W 22 ) < 0 , proving that − I + ˆ W 22 is Hurwitz. Since σ is arbitrary , − I + W is totally Hurwitz. (iv) The result follows from Lemma II.2(ii). Remark II.4. (Counterexamples for con verses of Lemma II.3). The conv erse of the implications in Lemma II.3 do not hold, as shown in the following. First, for a general matrix W , neither of ρ ( | W | ) and k W k is bounded by the other . The former is larger for [8 , 3; 2 , − 1] , e.g., while the latter is larger for [0 , 0; 1 , 0] . Howe ver , if W satisﬁes the Dale’ s law (as many biological neuronal networks do), i.e., each column is either nonnegati ve or nonpositive, then kj W jk < 1 k W k < 1 ρ ( j W j ) < 1 W 2 L I + W 2 H _ ρ ( [ W ] ) < 1 + I W 2 P _ Fig. 1: Inclusion relationships between the matrix classes introduced in Deﬁnition II.1 (cf. Lemma II.3). W = | W | D where D is a diagonal matrix such that | D | = I . Then, k W k = k| W |k ≥ ρ ( | W | ) , showing that, in this case, ρ ( | W | ) < 1 is a less restricti ve condition. Further , − I + W ∈ H 6⇒ ρ ( | W | ) < 1 as seen, e.g., from W = − 2 I . The same example shows W ∈ L 6⇒ k W k < 1 . Likewise, − I + W ∈ H 6⇒ W ∈ L , for which [0 . 5 , − 3; 4 , − 1] serves as a counter example (note that W ∈ L is an LMI feasibility problem that can be checked using standard solvers such as MA TLAB feasp function). Finally , A = [ − 1 , − 5 , 0; 0 , − 1 , − 6; − 1 , 0 , − 1] ensures the con verse of Lemma II.3(iv) does not hold either .  Figure 1 shows a summary of Lemma II.3 and Remark II.4. Dynamical Rate Models of Brain Networks Here we brieﬂy re view , following [46, §7], the construction of the linear-threshold network model used throughout the paper . In a lumped model, neurons are the smallest unit of neuronal circuits and the (directional) transmission of activity from one neuron to another takes place at a synapse , thus the terms pr e-synaptic and post-synaptic for the two neurons, respectiv ely . Both the input and output signals mainly consist of a sequence of spikes (action-potentials, Figure 2 top panel) which are modeled as impulse trains of the form ρ ( t ) = X k δ ( t − t k ) , where δ ( · ) denotes the Dirac delta function. In many brain areas, the exact timing { t k } of ρ ( t ) seems highly random while the ﬁring rate (number of spikes per second, Figure 2 bottom panel) sho ws greater trial-to-trial reproducibility . Therefore, a standard approximation is to model ρ ( t ) as the sample path of an inhomogeneous Poisson point process with rate, say , x ( t ) . Now , consider a pair of pre- and post-synaptic neurons with rates x pre ( t ) and x post ( t ) , respectiv ely . As a result of x pre ( t ) , an electrical current I post ( t ) ﬂows in the post-synaptic neuron. As- suming fast synaptic dynamics, I post ( t ) ∝ x pre ( t ) . Let w post,pre be the proportionality constant, so I post ( t ) = w post,pre x pre ( t ) . The pr e-synaptic neur on is called excitatory if w post,pre > 0 and inhibitory if w post,pre < 0 . In other words, excitatory neurons increase the activity of their out-neighbors while inhibitory neurons decrease it. 3 If the post-synaptic neuron receives input 3 While many brain networks, such as mammalian cortical networks, satisfy the Dale’ s law (cf. Remark II.4), all of our results in this work are applicable to arbitrary synaptic sign patterns. Fig. 2: A sample intracellular recording illustrating the spike train used for neuronal communication (top panel, measured intracellu- larly [47], [48]) and the corresponding (estimate of) ﬁring rate (bot- tom panel, estimated by binning spikes in 100 ms bins and smoothing with Gaussian window with 500 ms standard deviation). from multiple neurons, I post ( t ) follows a superposition law , I post ( t ) = X j w post ,j x j ( t ) , (1) where the sum is taken over its in-neighbors. If I post is constant, the post-synaptic rate approximately follows x post = F ( I post ) , where F is a nonlinear “response function”. Among the tw o widely used response functions, sig- moidal and linear-threshold, we use the latter for its analytical tractability: F ( · ) = [ · ] m post 0 . Finally , if I post ( t ) is time-varying, x post ( t ) “lags” F ( I post ( t )) with a time constant τ , i.e., τ ˙ x post ( t ) = − x post ( t ) + [ I post ( t )] m post 0 . (2) Equations (1)-(2) are the basis for our network model de- scribed next. I I I . P RO B L E M F O R M U L A T I O N Consider a network of neurons ev olving according to (1)- (2). Since the number of neurons in a brain region is very large, it is common to consider a population of neur ons with similar activ ation patterns as a single node with the a verage ﬁring rate of its neurons. This con vention also has the advantage of getting more consistent rates, as the ﬁring pattern of individual neurons may be sparse. 4 Combining the nodal rates in a vector x ∈ R n and synaptic weights in a matrix W ∈ R n × n , we obtain, according to (1)-(2), the linear-thr eshold network dynamics τ ˙ x ( t ) = − x ( t ) + [ Wx ( t ) + d ( t )] m 0 , 0 ≤ x (0) ≤ m , (3) m ∈ R n > 0 ∪ {∞} n . The term d ( t ) ∈ R n captures the external inputs to the net- work, including un-modeled background activity and possibly nonzero thresholds (i.e., if a node i becomes active when ( Wx + d ) i > ϑ i for some threshold ϑ i 6 = 0 ). The v ector of state upper bounds m can be ﬁnite ( m ∈ R n > 0 ) or inﬁnite ( m = ∞ 1 n ). Even though all biological neurons ev entually saturate for high input values, whether ﬁnite or inﬁnite m gives a more realistic/appropriate model can vary from brain re gion to brain region depending on whether typical 4 Our discussion is ne vertheless valid irrespective of whether network nodes represent individual neurons or groups of them. . . . . . . Subnetwork i − 1 Subnetwork i Subnetwork i + 1 N 0 i (inhibited) N 1 i (recruited) Fig. 3: The hierarchical network structure considered in this work. Each layer is only directly connected to the layers belo w and abov e it. Longer-range connections between non-successive layers do e xist in thalamocortical hierarchies but are weaker than those between successiv e layers and are not considered in this work for simplicity . (in vi vo) v alues of x reach (near) saturation. Historically , the unbounded case has in fact been used and studied more extensi vely , both in computational neuroscience and machine learning. See, e.g., [49] and the references therein for evidence in fa vor of unbounded acti v ation functions. Surprisingly , ho w- ev er , the analytical properties of the two cases are very similar , as we will see throughout this work. Further , note that the right-hand side of (3) is globally Lipschitz-continuous (though not smooth) and therefore a unique continuously dif ferentiable solution exists for all t ≥ 0 [50, Thm 3.2]. Moreover , it is straightforward to show that if 0 ≤ x (0) ≤ m then 0 ≤ x ( t ) ≤ m for all t ≥ 0 . Consistent with the vision for hierarchical selectiv e recruit- ment (HSR) outlined in Section I, we consider a hierarchy of linear-threshold networks of the form (3), as depicted in Figure 3. For each layer i , we use N i , N 1 i , and N 0 i , i ∈ { 1 , . . . , N } to denote the corresponding subnetwork and its task-relev ant and task-irrele vant sub-subnetw orks, respectiv ely . Even when considered in isolation, each layer of the net- work exhibits rich dynamical behavior . In fact, simulations of (3) with random W and d reveal that – locally , the dynamics may ha ve zero, one, or many stable and/or unstable equilibrium points, – globally , the dynamics can exhibit nonlinear phenomena such as limit cycles, multi-stability , and chaos, – the state trajectories may grow unbounded (if m = ∞ 1 n ) if the excitatory subnetwork [ W ] ∞ 0 is sufﬁciently strong. This richness of behavior can only increase if layers are subject to time-varying inputs d ( t ) and, in particular , when interconnected with other layers in the hierarchy . Motiv ated by these observ ations, our ultimate goal in this work is to tackle four problems: (i) the analysis of the relationship between structure ( W ) and dynamical beha vior (basic properties such as existence and uniqueness of equilibria (EUE), asymptotic stability , and boundedness of trajectories) for each subnetwork when oper- ating in isolation from the rest of the network ( d ( t ) ≡ d ); (ii) the analysis of the conditions on the joint structure of each two successiv e layers N i and N i +1 that allows for selectiv e inhibition of N 0 i +1 by its input from N i , being equiv alent to the stabilization of N 0 i +1 to 0 (inactivity); (iii) the analysis of the conditions on the joint structure of each two successiv e layers N i and N i +1 that allows for top-down recruitment of N 1 i +1 by its input from N i , being equiv alent to the stabilization of N 1 i +1 tow ard a desired trajectory set by N i (activity); (iv) the combination of (ii) and (iii) in a uniﬁed frame work and its e xtension to the complete N -layer network of netw orks. Problems (i) and (ii) are the focus of this paper , whereas we address problems (iii) and (iv) in the accompanying work [19]. I V . I N T E R NA L D Y N A M I C S O F S I N G L E - L A Y E R N E T W O R K S In this section, we pro vide an in-depth study of the basic dynamical properties of the network dynamics (3) in isolation. In such case, the external input d ( t ) boils down to background activity and possibly nonzero thresholds, which are constant relativ e to the timescale τ . The dynamics (3) thus simplify to τ ˙ x ( t ) = − x ( t ) + [ Wx ( t ) + d ] m 0 , 0 ≤ x (0) ≤ m , (4) m ∈ R n > 0 ∪ {∞} n . In the following, we deri ve conditions in terms of the network structure for the existence and uniqueness of equilibria (EUE), asymptotic stability , and boundedness of trajectories. A. Dynamics as Switched Af ﬁne System The nonlinear dynamics (4) is a switched af ﬁne system with 2 n modes if m = ∞ 1 n or 3 n modes if m < ∞ 1 n . Each mode of this system corresponds to a switching index σ = σ ( x ) ∈ { 0 , `, s } n , where for each i ∈ { 1 , . . . , n } , σ i = 0 if the node is inactive (i.e., ( Wx + d ) i ≤ 0 ), σ i = ` if the node is in linear regime (i.e., ( Wx + d ) i ∈ [0 , m i ) ), and σ i = s if the node is saturated (i.e., ( Wx + d ) i ≥ m i ). Clearly , the mode of the system is state-dependent and each switching index σ ∈ { 0 , `, s } n corresponds to a switching region Ω σ = { x | ( Wx + d ) i ∈ ( −∞ , 0] , ∀ i s . t . σ i = 0 , and ( Wx + d ) i ∈ [0 , m i ] , ∀ i s . t . σ i = `, and ( Wx + d ) i ∈ [ m i , ∞ ) , ∀ i s . t . σ i = s } . W ithin each Ω σ , we hav e [ Wx ( t ) + d ] m 0 = Σ ` ( Wx ( t ) + d ) + Σ s m , where Σ ` = Σ ` ( σ ) is a diagonal matrix with Σ ` ii = 1 if σ i = ` and Σ ` ii = 0 otherwise. Σ s is deﬁned similarly , and we set the con vention that Σ s m = 0 if m = ∞ 1 n . Therefore, (4) can be written in the equiv alent piecewise-af ﬁne form τ ˙ x = ( − I + Σ ` W ) x + Σ ` d + Σ s m , ∀ x ∈ Ω σ . (5) This switched representation of the dynamics motiv ates the following assumptions on the weight matrix W . Assumption 1. Assume (i) det ( W ) 6 = 0 ; (ii) det ( I − Σ ` W ) 6 = 0 for all the 2 n matrices Σ ` ( σ ) , σ ∈ { 0 , `, s } n . Assumption 1 is not a restriction in practice since the set of matrices for which it is not satisﬁed can be e xpressed as a ﬁnite union of measure-zero sets, and hence has measure zero. By Assumption 1(i), the system of equations Wx + d = 0 deﬁnes a non-degenerate set of n hyperplanes partitioning R n into 2 n solid con vex polytopic translated cones apex ed at − W − 1 d . 5 Unlike linear systems, the existence of equilibria is not guaranteed for (5). Rather, each σ ∈ { 0 , `, s } n corresponds to an equilibrium candidate x ∗ σ = ( I − Σ ` W ) − 1 ( Σ ` d + Σ s m ) , (6) which is an equilibrium if it belongs to Ω σ . W e next identify conditions for this to be true. B. Existence and Uniqueness of Equilibria (EUE) The ﬁrst step in analyzing any dynamical system is the char- acterization of its equilibria. W e being our analysis of the EUE with the case of bounded activ ation functions ( m ∈ R n > 0 ). Theorem IV .1. (EUE). The network dynamics (4) has a unique equilibrium for all d ∈ R n if and only if I − W ∈ P . Pr oof: Despite their similarity , different equiv alences need to be established and results need to be in voked for the bounded and unbounded cases. Therefore, we prov e the result separately for each case. Case 1: m < ∞ 1 n . The existence of equilibria is guaran- teed by the Brouwer ﬁxed point theorem [52] for all W and all d . W e use results from [53] to characterize uniqueness. Follo wing the terminology therein, the set C = { Ω σ | σ ∈ { 0 , `, s } n } is a chamber system and its branching number is 4 by Assumption 1. Let f ( x ; d ) = x − [ Wx + d ] m 0 which, for any d , is piecewise-af ﬁne on the chamber system C by (5) and is proper since k f ( x ; d ) k → ∞ whene ver k x k → ∞ . First, assume that I − W ∈ P . Then, f is coherently oriented by deﬁnition and thus [53, Thm 5.3] ensures that f is bijectiv e. In particular, there exists a unique x such that f ( x ; d ) = 0 , giving uniqueness of equilibria for any d . Now , assume that I − W / ∈ P . Since the determinant of I − Σ ` W is always positi ve on the chamber Ω 0 , f cannot be coherently oriented, thus not bijective by [53, Thm 5.3], and thus not injecti ve by [53, Cor 5.2]. Therefore, there exists x 1 , x 2 , z ∈ R n such that x 1 6 = x 2 but x 1 − [ Wx + d ] m 0 = z = x 2 − [ Wx + d ] m 0 , where d ∈ R n is arbitrary . Therefore, f ( · ; Wz + d ) has two distinct roots, x 1 − z and x 2 − z , proving the necessity of I − W ∈ P for uniqueness of equilibria. Case 2: m = ∞ 1 n . In this case, we simply show the equi v a- lence between our equilibrium equation x = [ Wx + d ] ∞ 0 and the well-studied linear complementarity problem (LCP). By Assumption 1, x = [ Wx + d ] ∞ 0 ⇔ Wx + d = W [ Wx + d ] ∞ 0 + d . (7) 5 Recall that a set of n hyperplanes is non-deg enerate [51] if their in- tersection is a point or, equiv alently , the matrix composed of their normal vectors is nonsingular . A set S ⊆ R n is called a polytope if it has the form S = { x | Ax ≤ b } ; a cone if c x ∈ S for any x ∈ S, c ∈ R ≥ 0 ; a translated cone ape xed at y if { x | x + y ∈ S } is a cone; conve x if (1 − θ ) x + θ y ∈ S for any x , y ∈ S, θ ∈ [0 , 1] ; and solid if it has a non-empty interior. W e next perform a change of variables as follo ws. Let z = [ Wx + d ] ∞ 0 and w = [ − Wx − d ] ∞ 0 . These vectors ha ve the properties that z , w ∈ R n ≥ 0 , z T w = 0 , and Wx + d = z − w . and thus provide a unique (in vertible) characterization of x . Therefore, (7) is equiv alent to w = ( I − W ) z − d , z , w ∈ R n ≥ 0 , z T w = 0 , which is the standard LCP and has a unique solution ( z , w ) for all d ∈ R n if and only if I − W ∈ P [54]. Similar to Case 1, it can also be shown that if I − W / ∈ P , there exists at least one d for which two equilibrium points exists (see, e.g., the proof of [54, Thm 4.2]). This completes the proof. Even though the condition I − W ∈ P may seem abstract, it has a nice geometric interpretation. From [53, Lem 2.2], I − W ∈ P if and only if the (negativ e) vector ﬁeld x 7→ x − [ Wx + d ] m 0 maps each switching region Ω σ to another polytopic region and the images of adjacent switching regions remains adjacent. In other words, the vector ﬁeld has a coher ent orientation when mapping the state space. 6 Remark IV .2. (Computational complexity of verifying I − W ∈ P ). Although the problem of determining whether a matrix is in P is straightforward for small n , it is known to be co-NP-complete [56], and thus expensi ve for large networks. Indeed, [57] shows that all the 2 n principal minors of A hav e to be checked to prove A ∈ P (though disproving A ∈ P is usually much easier). In these cases, one may need to rely on more conservati ve sufﬁcient conditions such as ρ ( | W | ) < 1 or k W k < 1 (cf. Lemma II.3) to establish I − W ∈ P . These conditions, moreov er, ha ve the added beneﬁt of providing intu- itiv e connections between the distribution of synaptic weights, network size, and stability . W e elaborate more on this point in Section V -C.  Example IV .3. (W ilson-Co wan model). Consider a network of n nodes in which αn, α ∈ (0 , 1) are excitatory , (1 − α ) n are inhibitory . Under some regularity assumptions, giv en next, this network can be (further) reduced to a simple, two-dimensional network commonly known as the W ilson-Cowan model [58] and widely used in computational neuroscience [59], [60]. Assume that the synaptic weight between any pair of nodes, the external input to them, and their maximal ﬁring rate (if ﬁnite) only depends on their type: the synaptic weight of any inhibitory-to-excitatory connection is w ei < 0 , similarly for w ee > 0 , w ie > 0 , w ii < 0 , and all excitatory (inhibitory) nodes recei ve d e ∈ R ( d i ∈ R ) and hav e maximal rate m e ∈ R > 0 ∪ {∞} ( m i ∈ R > 0 ∪ {∞} ). Let x e ( t ) and x i ( t ) be the av erage ﬁring rates of excitatory and inhibitory nodes, respectiv ely . Then, (4) simpliﬁes to τ  ˙ x e ˙ x i  = −  x e x i  +  αnw ee (1 − α ) nw ei αnw ie (1 − α ) nw ii  x e x i  +  d e d i  m 0 . 6 A closely-related class of matrices is that of M-matrices [45] with established relationships with the stability of nonlinear systems, see, e.g., [55]. Let W E I ∈ R 2 × 2 be the corresponding weight matrix abo ve. One can check that I − W E I ∈ P ⇔ αnw ee < 1 , (8) and ρ ( | W E I | ) < 1 ⇔ αnw ee < 1 , (1 − α ) n | w ii | < 1 , and α (1 − α ) n 2 w ie | w ei | < (1 − αnw ee )(1 − (1 − α ) n | w ii | ) . Thus, according to Theorem IV .1, EUE only requires the excitatory dynamics to be stable (note that w ee has to become smaller as n grows), while the more conservati ve condition ρ ( | W E I | ) < 1 also requires (relativ ely) weak inhibitory- inhibitory synapses and a weak interconnection between exci- tatory and inhibitory subnetworks.  When I − W ∈ P , Theorem IV .1 ensures EUE for all d ∈ R n . When I − W / ∈ P , howe ver , a more inv olved question is to ﬁnd the values of d that giv e rise to non-unique equilibrium points. T o answer this, we next perform a more direct analysis of the equilibria. F or simplicity , we focus in the remainder of this section on unbounded dynamics ( m = ∞ 1 n ). Recall the deﬁnition of an equilibrium candidate in (6). Using Assumption 1, and after some manipulations, we hav e Wx ∗ σ + d = W ( I − Σ ` W ) − 1 Σ ` d + d (9) = ( W − 1 − Σ ` ) − 1 Σ ` d + d = ( I − WΣ ` ) − 1 WΣ ` d + d =  ( I − WΣ ` ) − 1 WΣ ` + I  d = ( I − W Σ ` ) − 1 d , thus, x ∗ σ ∈ Ω σ ⇔ (2 Σ ` − I )( I − WΣ ` ) − 1 | {z } , M σ d ≥ 0 . (10) Therefore, if M σ d ≥ 0 for exactly one σ ∈ { 0 , ` } n , then a unique equilibrium exists. Howe ver , when M σ ` d ≥ 0 for multiple σ ` ∈ { 0 , ` } n , ` ∈ { 1 , . . . , ¯ ` } , the network may hav e either multiple equilibria or a unique one x ∗ σ 1 = · · · = x ∗ σ ¯ ` lying on the boundary between { Ω σ ` } ¯ ` ` =1 . The next result shows that the quantities M σ d can be used to distinguish between these two latter cases. Lemma IV .4. (Existence of multiple equilibria). Assume W satisﬁes Assumption 1, d ∈ R n is arbitrary , and M σ is deﬁned as in (10) for σ ∈ { 0 , ` } n . If there exist σ 1 6 = σ 2 such that M σ 1 d ≥ 0 and M σ 2 d ≥ 0 , then x ∗ σ 1 = x ∗ σ 2 if and only if M σ 1 d = M σ 2 d . Pr oof: Clearly , x ∗ σ 1 = x ∗ σ 2 ⇔ Wx ∗ σ 1 + d = Wx ∗ σ 2 + d ⇔ ( I − W Σ 1 ) − 1 d = ( I − W Σ 2 ) − 1 d , (11) where we ha ve used (9). Since both M σ 1 d and M σ 2 d are nonnegati ve, (11) holds if and only if  ( I − W Σ 1 ) − 1 d  i =  ( I − W Σ 2 ) − 1 d  i = 0 for any i such that σ 1 ,i 6 = σ 2 ,i , which is equiv alent to M σ 1 d = M σ 2 d . This property of M σ can be used to deri ve a computa- tionally more in volv ed but input-dependent characterization of EUE, as follows. Proposition IV .5. (Optimization-based condition for EUE). Let W satisfy Assumption 1 and M σ be as deﬁned in (10) for σ ∈ { 0 , ` } n . F or d ∈ R n , deﬁne µ 1 ( d ) and µ 2 ( d ) to be the lar gest and second largest elements of the set  min i =1 ,...,n ( M σ d ) i | σ ∈ { 0 , ` } n  , r espectively . Then, (4) has a unique equilibrium for eac h d ∈ R n if and only if max k d k =1 µ 1 ( d ) µ 2 ( d ) < 0 . (12) Pr oof: First, note that d = 0 is a degenerate case where the origin is the unique equilibrium belonging to all Ω σ . For any d 6 = 0 and σ ∈ { 0 , ` } n , M σ d ≥ 0 if and only if M σ d / k d k ≥ 0 . Thus, EUE for all d ∈ R n and all k d k = 1 are equiv alent. Then, for any d , µ 1 ( d ) µ 2 ( d ) < 0 ⇔ µ 1 ( d ) > 0 and µ 2 ( d ) < 0 ⇔ ∃ unique M σ d ≥ 0 , σ ∈ { 0 , ` } n . (13) Note that the latter allows for the possibility of the existence of multiple σ with M σ d ≥ 0 provided that they hav e the same value of M σ d . By Lemma IV .4, (13) is then equi valent to EUE, completing the proof. In our experience, the optimization in volved in (12) is usu- ally highly non-conv ex but since the search space {k d k = 1 } is compact, global search methods can be used to verify (12) numerically if n is not too large. Ho wever , note that our main interest in (12) (being equi v alent to I − W ∈ P ) is when it does not hold. If so, then any d for which (12) fails gives a ray { α d | α > 0 } of input values that gi ve rise to non-unique equilibria. Combined with stability analysis of Section IV -C, e.g., this can be a basis for the characterization of multistability in linear-threshold dynamics which is itself beyond the scope of this work. The proof of Theorem IV .1 (for the unbounded case) is based on the LCP , which makes the relationship between I − W ∈ P and EUE opaque, even when taking into account the proof of the LCP . The equilibrium characterization in (10), howe ver , can be used to explain this relationship more transparently . For any giv en d , the non-uniqueness of equilibria is equiv alent to asking whether ∃ σ 1 , σ 2 ∈ { 0 , ` } n s.t. M σ 1 d ≥ 0 and M σ 2 d ≥ 0 M σ 1 d 6 = M σ 2 d , or , whether there exist q σ 1 6 = q σ 2 ∈ R n ≥ 0 such that M − 1 σ 1 q σ 1 = M − 1 σ 2 q σ 2 = d . A more general question, which turns out to be relev ant to EUE, is whether ∃ q σ 1 6 = q σ 2 ∈ O n s.t. q σ 1 = M σ 1 M − 1 σ 2 q σ 2 , (14) for any orthant O n of R n (including O n = R n ≥ 0 as a special case). This depends on whether the matrix M σ 1 M − 1 σ 2 can map any nonzero vector to the same orthant which, by Lemma II.2(iii), happens if and only if − M σ 1 M − 1 σ 2 / ∈ P . The follo wing result, whose proof is in Appendix A, giv es a necessary and sufﬁcient condition for this to not happen. Theorem IV .6. (Coherently oriented vector ﬁelds and the validity of equilibrium candidates). Let W satisfy Assump- tion 1 and M σ be deﬁned as in (10) . Then, − M σ 1 M − 1 σ 2 ∈ P for all (distinct) σ 1 , σ 2 ∈ { 0 , ` } n if and only if I − W ∈ P . Theorem IV .6 provides a more transparent account of the relationship between I − W ∈ P and EUE. If I − W ∈ P , then Theorem IV .6 and Lemma II.2(iii) ensure that none of M σ 1 M − 1 σ 2 can map a vector to the same orthant. Thus, no two q σ = M σ d belong to the same orthant. Therefore, there exists a one-to-one correspondence between { q σ } and orthants in R n , ensuring that exactly one q σ belongs to R n ≥ 0 , i.e., EUE. 7 W e end this subsection with a result that bounds the number and location if equilibria for the case when I − W / ∈ P . For A ∈ R n × n and σ ∈ { 0 , ` } n , let A ( σ ) be the principal submatrix of A containing the rows and columns for which σ i = ` . Further , for σ 1 , σ 2 ∈ { 0 , ` } n , we say σ 1 ≤ σ 2 if σ 1 ,i = ` ⇒ σ 2 ,i = ` for all i ∈ { 1 , . . . , n } . Corollary IV .7. (P artial EUE). Consider the dynamics (4) and assume that Assumption 1 holds. If I − W ( ¯ σ ) ∈ P for any ¯ σ ∈ { 0 , ` } n , then S σ ≤ ¯ σ Ω σ contains at most one equilibrium point. Pr oof: The proof follo ws directly from the proof of Theorem IV .6 and the fact that, using the deﬁnitions therein, − Γ ∈ P only requires I − W ([ ` n 1 + n 2 + n 3 0 n 4 ] T ) ∈ P . Even in the simplest case when I − W ∈ P , the resulting unique equilibrium may or may not be stable, as studied next. C. Asymptotic Stability The EUE is an opportunity to shape the network acti vity at steady state, provided that the equilibrium corresponds to a desired state (a memory , the encoding of a spatial location, eye position, etc. [61]–[65]) and it attracts netw ork trajectories. Here we in vestigate the latter . Theorem IV .8. (Asymptotic Stability). Consider the network dynamics (4) and assume W satisﬁes Assumption 1. (i) [Suf ﬁcient condition] If W ∈ L , then for all d ∈ R n , the network is globally exponentially stable (GES) relative to a unique equilibrium x ∗ ; (ii) [Necessary condition] If for all d ∈ R n the network is locally asymptotically stable r elative to a unique equilib- rium x ∗ , then − I + W ∈ H . Pr oof: (i) The EUE follows from Lemma II.3(iii)&(iv) and Theorem IV .1. GES can be deduced from [66, Thm 1], but a simpler and direct proof can also be found in a preliminary version of this w ork [67] which is omitted here space reasons. (ii) Assume, by contradiction, that − I + W / ∈ H , which means that there exists σ ∈ { 0 , ` } n such that − I + Σ ` W is not Hurwitz. Let σ 01 ∈ { 0 , 1 } n hav e the same zeros as σ , and consider the choice d = (2 I − W ) σ 01 − 1 n . It is straightforward to show that x ∗ = σ 01 is an equilibrium point for (4) lying in the interior of Ω σ . By assumption, x ∗ is 7 W ith a careful resolution of the ties, this still holds in the measure-zero ev ent that multiple q σ are equal and belong to the boundary between orthants. (unique and) locally asymptotically stable, which contradicts − I + Σ ` W not being Hurwitz. This completes the proof. Similar to the problem of verifying whether a matrix is a P-matrix, cf. Remark IV .2, the computational complexity of verifying total-Hurwitzness grows exponentially with n . The same applies to the veriﬁcation of total L -stability , see, e.g., [68] and the references therein. The next result gi ves a usually more conserv ati ve b ut computationally inexpensiv e alternativ e. Proposition IV .9. (Computationally feasible sufﬁcient con- ditions for GES). Consider the network dynamics (4) and assume the weight matrix W satisﬁes Assumption 1. If ρ ( | W | ) < 1 or k W k < 1 , then for all d ∈ R n , the network has a unique equilibrium x ∗ and it is GES relative to x ∗ . Pr oof: If k W k < 1 , the result follows from Lemma II.3(ii) and Theorem IV .8. For the case ρ ( | W | ) < 1 , the same proof technique as in [25, Prop. 3] can be used to prov e GES, as shown in a preliminary version of this work [67], but is omitted here for space reasons. From Lemma II.3(iii), the conditions of Theorem IV .8 and Proposition IV .9 are not conclusi ve when W satisﬁes − I + W ∈ H but neither W ∈ L nor ρ ( | W | ) < 1 . Ho wever , (i) If a unique equilibrium x ∗ lies in the interior of a switching region (a condition that can be shown to hold for almost all d ), then x ∗ is at least locally exponentially stable (since a sufﬁciently small region of attraction of x ∗ is contained in that switching region). (ii) In our extensiv e simulations with random ( W , d ) , any system satisfying − I + W ∈ H was GES for all d . These observ ations lead us to the follo wing conjecture, whose analytic characterization remains an open problem. Conjecture IV .10. (Sufﬁciency of total-Hurwitzness for GES). Consider the dynamics (4) and assume W satisﬁes Assumption 1. The network has a unique GES equilibrium for all d ∈ R n if and only if − I + W ∈ H . W e next study the GES of the uniform excitatory-inhibitory networks of Example IV .3. Example IV .11. (W ilson-Cowan model, cont’d). Consider the W ilson-Cowan model of Example IV .3. One can verify − I + W E I ∈ H ⇔ αnw ee < 1 , (15) thus being equiv alent (in this two-dimensional case) to I − W E I ∈ P and, interestingly , only restricting w ee while w ei , w ie , and w ii are completely free. Figure 4 shows sample phase portraits for the cases αnw ee < 1 and αnw ee > 1 , matching our expectations from Theorems IV .1 and IV .8. While our focus here is on the e xistence, uniqueness, and stability of equilibria, it is instructive to highlight the role of equilibrium analysis and, in particular , lac k of stable equilibria in the generation of oscillations in the same linear -threshold Wilson- Cow an model [69]. In this case, both the linear-threshold W ilson-Cowan model and the popular Kuramoto model [70]– [72] of neural oscillations provide parallel simpliﬁcations to the (more biologically accurate) Wilson-Co wan model with smooth sigmoidal nonlinearities, cf. [73].  (a) (b) Fig. 4: Network trajectories for the excitatory-inhibitory network of Example IV .11. (a) When W E I = [0 . 9 , − 2; 5 , − 1 . 5] , d E I = [1; 1] , the network has a unique GES equilibrium. (b) Howe ver , for W E I = [1 . 1 , − 2; 5 , − 1 . 5] , d E I = [ − 0 . 01; − 1] , the network exhibits bi-stable beha vior . m = ∞ 1 2 in both cases. The trajectory colors correspond to the equilibria to which they conv erge. Although αnw ee > 1 , the network is GES for most values of d E I , so we used Proposition IV .5 for ﬁnding a d E I that leads to multi-stability . D. Boundedness of Solutions Here we study the boundedness of solutions under the dynamics (3). While our discussion so far has been about (4) (with constant d ), we switch for the remainder of this section to (3) for the sake of generality , as the same results are appli- cable without major modiﬁcations. Since network trajectories are trivially bounded if m < ∞ 1 n , we limit our discussion here to the unbounded case. Note that in reality , the ﬁring rate of any neuron is bounded by a maximum rate dictated by its refractory period (the minimum inter-spike duration). Unboundedness of solutions in this model corresponds in practice to the so-called “run-aw ay” e xcitations where the ﬁring of neurons gro w beyond sustainable rates for prolonged periods of time, which is neither desirable nor safe [74]. Since GES implies boundedness of solutions, any condition that is suf ﬁcient for GES is also sufﬁcient for boundedness. Howe ver , boundedness of solutions can be guaranteed under less restricti ve conditions. The next result shows that inhibi- tion, ov erall, preserves boundedness. Lemma IV .12. (Inhibition preserves boundedness). Let t 7→ x ( t ) be the solution of (3) starting from initial state x (0) = x 0 . Consider the system τ ˙ ¯ x ( t ) = − ¯ x ( t ) + [[ W ] ∞ 0 ¯ x ( t ) + d ( t )] ∞ 0 , ¯ x (0) = x 0 . (16) Then, x ( t ) ≤ ¯ x ( t ) for all t ≥ 0 . Pr oof: Since x ( t ) ≥ 0 for all t , we can write (3) as τ ˙ x ( t ) = − x ( t ) + [[ W ] ∞ 0 x ( t ) + d ( t ) + δ ( t )] ∞ 0 , (17) where δ ( t ) , ( W − [ W ] ∞ 0 ) x ( t ) ≤ 0 . Since the vector ﬁeld ( x , t ) 7→ − x + [[ W ] ∞ 0 x + d ( t )] ∞ 0 is quasi-monotone nonde- creasing 8 , the result follo ws by using the monotonicity of [ · ] ∞ 0 and applying the vector-v alued extension of the Comparison Principle giv en in [75, Lemma 3.4] to (16) and (17). While the result about preservation of boundedness under inhibition in Lemma IV .12 is intuitive, one must interpret it carefully: it is not in general true that adding inhibition to any dynamics (3) can only decrease x ( t ) . This is only true if the network vector ﬁeld is quasi-monotone nondecreasing, as is the case with the excitatory-only dynamics (16). Intuitively , this is because, if the network has inhibitory nodes, adding inhibition to their input can in turn “disinhibit” and increase the activity of the rest of the network. The next result identiﬁes a condition on the excitatory part of the dynamics to determine if trajectories are bounded. Theorem IV .13. (Boundedness). Consider the network dy- namics (3) . If the corr esponding excitatory-only dynamics (16) has bounded trajectories, the trajectories of (3) are also bounded by the same bound as those of (16) . The proof of this result follows from Lemma IV .12 and is therefore omitted. The following result, similar to Propo- sition IV .9, provides a more conservati ve but computationally feasible test for boundedness. Corollary IV .14. (Boundedness). Consider the network dy- namics (3) and assume that d ( t ) is bounded, i.e., ther e exists ¯ d ∈ R n > 0 such that d ( t ) ≤ ¯ d , t ≥ 0 . If ρ ([ W ] ∞ 0 ) < 1 , then the network trajectories r emain bounded for all t ≥ 0 . Pr oof: If d ( t ) is constant, the result follo ws from The- orem IV .13 and Proposition IV .9. If d ( t ) is not constant, the same argument prov es boundedness of trajectories for τ ˙ ¯ x ( t ) = − ¯ x ( t ) + [[ W ] ∞ 0 ¯ x ( t ) + ¯ d ] ∞ 0 , ¯ x (0) = x 0 . (18) The result then follo ws from the quasi-monotonicity of ( x , t ) 7→ − x + [ W + x + ¯ d ] ∞ 0 , similar to Lemma IV .12. Example IV .15. (Uniform excitatory-inhibitory networks, cont’d). Let us revisit the excitatory-inhibitory network of Example IV .3, here with m = ∞ 1 2 . Clearly , the excitatory- only dynamics hav e bounded trajectories if and only if ρ ([ W E I ] ∞ 0 ) < 1 ⇔ αnw ee < 1 , (19) which is the same condition as in (15) and (8). Ho we ver , an exhausti ve inspection of the switching regions { Ω σ } σ and the 8 A vector ﬁeld f : R n × R → R n is quasi-monotone nondecreasing [75, Def 2.3] if for any x , y ∈ R n and any i ∈ { 1 , . . . , n } ,  x i = y i and x j ≤ y j for all j 6 = i  ⇒ f ( x , t ) ≤ f ( y , t ) . eigen values of {− I + Σ ` W } σ rev eals that the boundedness of trajectories can also be guaranteed with the weaker condition − I + W be Hurwitz ⇔ ( (1 − αnw ee ) + (1 − (1 − α ) nw ii ) > 0 , and (1 − αnw ee )(1 − (1 − α ) nw ii ) > α (1 − α ) n 2 w ie w ei , showing that there is room for sharpening Theorem IV .13.  Remark IV .16. (Comparison with the literature). In this section, we hav e provided a comprehensive list of conditions that both extend and simplify the state of the art on stability of dynamically isolated linear-threshold networks. T o the best of our kno wledge, all the results are nov el for the bounded case with the exception of the sufﬁcienc y of I − W ∈ P for the uniqueness of equilibria (one of the four implications in Theorem IV .1) sho wn in [26] and Theorem IV .8(i), for which we present a simpler proof. Regarding unbounded networks, for equilibria we have e xtended [53, Thm 5.3] (implying only the sufﬁciency of I − W ∈ P for EUE) to show both necessity and sufﬁciency of I − W ∈ P for both existence and unique- ness (Theorem IV .1) and provided se veral results that partially characterize equilibria when I − W / ∈ P . On exponential stability of the unbounded case, Theorem IV .8 giv es a simpler proof than [66, Thm 1] for the sufﬁcienc y of W ∈ L and a nov el proof for the necessity of − I + W ∈ H . Finally , our result on boundedness of trajectories (Theorem IV .13) e xtends Corollary IV .14 (also av ailable in [34, Thm 1]) to a much wider class of networks by exploiting the quasi-monotonicity of excitatory-only dynamics.  V . S E L E C T I V E I N H I B I T I O N I N B I L A Y E R N E T W O R K S Here, we study selecti ve inhibition in bilayer networks as a building block towards the understanding of hierarchical selectiv e recruitment in multilayer networks. W ith respect to the model described in Section III, we consider two layers ( N = 2 ), where the dynamics of the lower layer N 2 is described by (3) and the dynamics of the upper layer N 1 is arbitrary (this is for generality , we consider linear-threshold dynamics for N 1 too in a multilayer framework in our ac- companying work [19]). Our goal is to study the selective inhibition of N 0 2 via the input that it receives from N 1 . As pointed out in Section III, when a group of neurons are inhibited, their activity is substantially decreased, ide- ally such that their net input (their respecti ve component of Wx ( t ) + d ( t ) ) becomes negati ve and their ﬁring rate decays exponentially to zero. Therefore, the problem of selective inhibition is equiv alent to the exponential stabilization of the nodes N 0 2 to the origin. T o this end, we decompose d ( t ) as d ( t ) = Bu ( t ) + ˜ d . (20) The role of u ( t ) ∈ R p ≥ 0 is to stabilize N 0 2 to the origin while the role of ˜ d ∈ R n is to shape the activity of N 1 2 by determining its equilibrium. For the purpose of this section, we assume ˜ d is giv en and constant. Let r ≤ n be the size of N 0 2 . W e partition x , W , B , and ˜ d accordingly , x =  x 0 x 1  , W =  W 00 W 01 W 10 W 11  , B =  B 0 0  , ˜ d =  0 ˜ d 1  , (21) where W 00 ∈ R r × r , B 0 ∈ R r × p ≤ 0 is nonpositiv e to deliver inhibition, and ˜ d 1 ∈ R n − r . m is decomposed similarly . The ﬁrst r rows of B are nonzero to allo w for the inhibition of N 0 2 while the remaining n − r rows are zero to make this inhibition selectiv e to N 0 2 . 9 The sparsity of the entries of ˜ d is opposite to the ro ws of B due to the complementary roles of Bu ( t ) and ˜ d . The mechanisms of inhibition in the brain are broadly divided [76] into two cate gories, feedforward and feedback, based on ho w the signal u ( t ) is determined. In the follo wing, we separately study each scenario, analyzing the interplay between the corresponding mechanism and network structure. W e will later combine both mechanisms when we discuss the complete HSR framework in [19], as natural selectiv e inhibition is not purely feedback or feedforward. A. F eedforwar d Selective Inhibition Feedforward inhibition [76] refers to the scenario where N 1 provides an inhibition based on its own “desired” activ- ity/inactivity pattern for N 2 and irrespecti ve of the current state of N 2 . This is indeed possible if the inhibition is sufﬁciently strong, as excessi ve inhibition has no effect on nodal dynamics due to the (negati ve) thresholding in [ · ] m 0 . Howe ver , this independence from the activity le vel of N 2 requires some form of guaranteed boundedness, as deﬁned next. Deﬁnition V .1. (Monotone boundedness). The dynamics (3) is monotonically bounded if for any ¯ d ∈ R n ther e exists ν ( ¯ d ) ∈ R n ≥ 0 such that x ( t ) ≤ ν ( ¯ d ) , t ≥ 0 for any d ( t ) ≤ ¯ d , t ≥ 0 . From Lemma IV .12 and Proposition IV .9, (3) is monoton- ically bounded if ρ ([ W ] ∞ 0 ) < 1 and the initial condition x 0 is restricted to a bounded domain. Also in reality , the state of any biological neuronal network is uniformly bounded due to the refractory period of its neurons, implying monotone boundedness. W e next show that the GES of N 1 2 is both necessary and sufﬁcient for feedforward selective inhibition. Theorem V .2. (F eedforward selective inhibition). Consider the dynamics (3) , where the external input is given by (20) - (21) with a constant feedforward control u ( t ) ≡ u ≥ 0 . Assume that (3) is monotonically bounded and range ([ W 00 W 01 ]) ⊆ range ( B 0 ) . (22) Then, for any ˜ d 1 ∈ R n − r , ther e exists ¯ u ∈ R p ≥ 0 such that for all u ≥ ¯ u , N 2 is GES r elative to a unique equilibrium of the form x ∗ = [ 0 T r ( x 1 ∗ ) T ] T if and only if W 11 is such that the internal N 1 2 dynamics τ ˙ x 1 = − x 1 + [ W 11 x 1 + ˜ d 1 ] m 1 0 , (23) is GES r elative to a unique equilibrium. Pr oof: ( ⇐ ) Deﬁne u s to be a solution of B 0 u s = − [[ W 00 W 01 ]] ∞ 0 ν ( ˜ d ) . (24) 9 This sparsity pattern can always be achiev ed by (re-)labeling the r directly controlled nodes as 1 , . . . , r , so that the n − r last entries of B are 0 . This solution exists by assumption (22). Let ¯ u = [ u s ] ∞ 0 and note that B 0 u ≤ B 0 ¯ u ≤ B 0 u s . By construction, (3), (20), (21), (24) simplify to τ ˙ x 0 = − x 0 , τ ˙ x 1 = − x 1 + [ W 10 x 0 + W 11 x 1 + ˜ d 1 ] m 1 0 , (25) whose GES follows from Lemma A.1. ( ⇒ ) By monotone boundedness and nonpositivity of B 0 , x ( t ) ≤ ν ( ˜ d ) for all t ≥ 0 and any u ≥ ¯ u . Let u = ¯ u + [ u s ] ∞ 0 where u s is a solution to (24). Similar to abov e, this simpliﬁes (3), (20), (21), (24) to (25), which is GES by assumption. Howe ver , for any initial condition of the form x (0) = [ 0 T r x 1 (0) T ] T , the trajectories of (25) are the same as (23), and the result follows. The next section shows that the condition (22) on the ability to inﬂuence the dynamics of the task-irrelev ant nodes through control also plays a ke y role in feedback selecti ve inhibition. W e defer the discussion about the interpretation of this condition to Section V -C below . B. F eedback Selective Inhibition The core idea of feedback inhibition [76], as found through- out the brain, is the dependence of the amount of inhibition on the activity level of the nodes that are to be inhibited. This dependence is in particular relev ant to GDSA, as the stronger and more salient a source of distraction, the harder one must try to suppress its effects on perception. The next result provides a nov el characterization of sev eral equiv alences between the dynamical properties of N 2 under linear full-state feedback inhibition and those of N 1 2 . Theorem V .3. (F eedback selective inhibition). Consider the dynamics (3) , wher e the external input is given by (20) - (21) with a linear state feedback u u ( t ) = Kx ( t ) , (26) and K ∈ R p × n is a constant control gain. Assume that (22) holds. Then, ther e almost always exists K ∈ R p × n such that (i) I − ( W + BK ) ∈ P if and only if I − W 11 ∈ P ; (ii) − I + ( W + BK ) ∈ H if and only if − I + W 11 ∈ H ; (iii) W + BK ∈ L if and only if W 11 ∈ L ; (iv) ρ ( | W + BK | ) < 1 if and only if ρ ( | W 11 | ) < 1 ; (v) k W + BK k < 1 if and only if k [ W 10 W 11 ] k < 1 . Pr oof: (i) ⇒ ) For any K = [ K 0 K 1 ] ∈ R p × n , W + BK =  W 00 + B 0 K 0 W 01 + B 0 K 1 W 10 W 11  . (27) Thus, since any principal submatrix of a P-matrix is a P- matrix, I − W 11 ∈ P . ⇐ ) By (22) there exists ¯ K ∈ R p × n such that −  W 00 W 01  = B 0 ¯ K . (28) Using the fact that the determinant of any block-triangular matrix is the product of the determinants of the blocks on its diagonal [45, Prop 2.8.1], it follows that I − ( W + B ¯ K ) ∈ P . (ii) ⇒ ) This follo ws from (27) and the fact that a principal submatrix of a totally-Hurwitz matrix is totally-Hurwitz. ⇐ ) Using the matrix ¯ K in (28), the result follows from the fact that the eigen values of a block-triangular matrix are the eigen values of its diagonal blocks. (iii) ⇒ ) Let P = P T > 0 be such that ( − I + ( W + BK ) T Σ ` ) P + P ( − I + Σ ` ( W + BK )) < 0 (29) for all σ ∈ { 0 , ` } n . Consider , in particular , σ = [ 0 T r ( σ 1 ) T ] T where σ 1 ∈ { 0 , ` } n − r is arbitrary . Let Σ ` 1 ∈ R ( n − r ) × ( n − r ) be the bottom-right block of Σ ` and partition P in 2 -by- 2 block form similarly to W . Since ( − I + ( W + BK ) T Σ ` ) P + P ( − I + Σ ` ( W + BK )) =  ? ? ? ( − I + Σ ` 1 W 11 ) T P 11 + P 11 ( − I + Σ ` 1 W 11 )  , and any principal submatrix of a negativ e deﬁnite matrix is negati ve deﬁnite, we deduce W 11 ∈ L . ⇐ ) Let P 11 ∈ R ( n − r ) × ( n − r ) be such that ( − I + ( W 11 ) T Σ ` 1 ) P 11 + P 11 ( − I + Σ ` 1 W 11 ) < 0 , for all σ 1 ∈ { 0 , 1 } n − r and ¯ K be as in (28). For any σ = [( σ 0 ) T ( σ 1 ) T ] T , (28) giv es − I + Σ ` ( W + B ¯ K ) =  − I 0 ? − I + Σ ` 1 W 11  . Thus, the dynamics τ ˙ x =  − I + Σ ` ( W + B ¯ K )  x is a cascade of τ ˙ x 0 = − x 0 and τ ˙ x 1 = ( − I + Σ ` 1 W 11 ) x 1 + ? · x 0 , where the latter has the ISS 10 -L yapunov function V 1 ( x 1 ) = ( x 1 ) T P 11 x 1 . Using [77, Thm 3], (29) holds for K = ¯ K , P = diag ( I , P 11 ) , and any σ ∈ { 0 , ` } n , giving W + B ¯ K ∈ L . (iv) ⇒ ) This follows from (27) and [45, Fact 4.11.19]. ⇐ ) Consider the matrix ¯ K in (28). The result then follows from the fact that the eigen values of a block-triangular matrix are the eigen values of its diagonal blocks. (v) ⇒ ) Note that for any K ∈ R p × n , k W + BK k 2 = ρ  ? ? ? W 10 ( W 10 ) T + W 11 ( W 11 ) T  ≥ ρ ( W 10 ( W 10 ) T + W 11 ( W 11 ) T ) =    W 10 W 11    2 , where the inequality follows from the well-known interlacing property of eigen values of principal submatrices (cf. [78]). ⇐ ) Consider the matrix ¯ K in (28) and note that k W + B ¯ K k 2 = ρ  0 0 0 W 10 ( W 10 ) T + W 11 ( W 11 ) T  = ρ ( W 10 ( W 10 ) T + W 11 ( W 11 ) T ) =    W 10 W 11    2 < 1 , completing the proof. Remark V .4. (F eedback inhibition with nonnegative u ( t ) ). Even though Theorem V .3 is motiv ated by feedback inhibition in the brain, the result illustrates some fundamental properties of linear -threshold dynamics and the corresponding matrix classes that is of independent interest, which motiv ates the generality of its formulation. The particular application to brain networks requires nonneg ati ve inputs, which we discuss next. The core principle of Theorem V .3 is the cancellation of 10 Input-to-state stability local input [ W 00 W 01 ] x to N 0 2 with the top-down feedback input B 0 ¯ Kx , simplifying the dynamics of N 0 2 to τ ˙ x 0 = − x 0 that guarantee its inhibition. Howe ver , the resulting input signal u = ¯ Kx (being the ﬁring rate of some neuronal population in N 1 ) may not remain nonnegati ve at all times. This can be easily addressed as follo ws. Similar to the proof of Theorem V .2, we let u ( t ) = [ ¯ Kx ( t )] ∞ 0 . This makes u ( t ) nonnegati ve without affecting the selectiv e inhibition of N 0 2 in (3) as B 0 ≤ 0 . In principle, a similar concern can exist as to whether ¯ Kx becomes larger than the maximum ﬁring rate of the corresponding populations in N 1 . Howe ver , this only relates to the magnitude of the entries in B 0 (via (28), as opposed to the sign of ¯ Kx , which relates to the sign of the entries in B 0 ), which can alw ays be increased via synaptic long term potentiation (L TP) [79], in turn decreasing the magnitude of the entries in ¯ K .  Remark V .5. (State vs. output feedback). The assumption of state feedback is a simplifying one and its generalization merits further research. Howe ver , we note that x 0 is most likely always available for feedback (as feedback inhibition is highly reciprocal at the neuronal le vel [80] and e ven more so at the population le vel) while the av ailability of x 1 for feedback remains case-speciﬁc. If the latter is not av ailable, one of two scenarios may happen: either the local interaction of x 0 and x 1 is competitiv e and W 01 ≤ 0 (which is not unlikely due to the pre v alence of lateral inhibition in the cortex [81]), in which case the feedback of x 1 is not e ven needed (similar to Remark V .4) or , at worst, the unobserved x 1 activ ely excites x 0 , in which case a combination of feedback and feedforward inhibition can be used, similar to our full model in Part II [19, Thm 4.3].  C. Network Size, W eight Distribution, and Stabilization Underlying the discussion abov e is the requirement that N 2 can be asymptotically stabilized towards an equilibrium which has some components equal to zero and the remaining components determined by ˜ d . Here, it is important to distin- guish between the stability of N 2 in the absence and presence of selecti ve inhibition. In reality , the large size of biological neuronal networks often leads to highly unstable dynamics if all the nodes in a layer , say N 2 , are acti ve. Therefore, the selecti ve inhibition of N 0 2 is not only responsible for the suppression of the task-irrele v ant acti vity of N 0 2 , but also for the overall stabilization of N 2 that allows for top-do wn recruitment of N 1 2 . This poses limitations on the size and structure of the subnetworks N 0 2 and N 1 2 . It is in this context that one can analyze the condition (22) assumed in both Theorems V .2 and V .3. This conditi on requires, essentially , that there are sufﬁciently man y “independent” external controls u to enforce inhibition on N 0 2 . The follo wing result formalizes this statement. Lemma V .6. (Equivalent characterization of (22) ). Let the matrices W 0 and B 0 have dimensions r × n and r × p , r espectively . Then, range ( W 0 ) ⊆ range ( B 0 ) for almost all ( W 0 , B 0 ) ∈ R r × n × R r × m if and only if p ≥ r . Pr oof: ⇒ ) Assume, by contradiction, that p < r , so range ( B 0 ) ( R r for any B 0 . Let Q = Q ( B 0 ) be a matrix whose columns form a basis for range ( B 0 ) ⊥ . Then, range ( W 0 ) ⊆ range ( B 0 ) if and only if Q ( B 0 ) T W 0 = 0 . By Fubini’ s theorem [82, Ch. 20], Z R r × n × R r × p I { Q ( B 0 ) T W 0 = 0 } ( W 0 , B 0 ) d ( W 0 , B 0 ) = Z R r × p d B 0 Z R r × n I { Q ( B 0 ) T W 0 = 0 } ( W 0 , B 0 ) d W 0 = Z R r × p 0 d B 0 = 0 , where I denotes the indicator function. This contradiction prov es p ≥ r . ⇐ ) Let B 0 = [ B 0 1 B 0 2 ] where B 0 1 ∈ R r × r . It is straightforward to show that { ( W 0 , B 0 ) | range ( W 0 ) * range ( B 0 ) } ⊆ R r × n × A, where A = { B 0 | det ( B 0 1 ) = 0 } . Since A has measure zero, the result follows from a similar argument as above inv oking Fubini’ s theorem. Based on intuitions from linear systems theory , it may be tempting to seek a relaxation of (22) for the case where p < r . This is due to the f act that for a linear system τ ˙ x = Wx + Bu , it is kno wn [83, eq (4.5) and Thm 3.5] that the set of all reachable states from the origin is given by range  B 0 W 00 B 0 · · · ( W n − 1 ) 00 B 0 0 W 10 B 0 · · · ( W n − 1 ) 10 B 0  , which is usually much larger than range ( B ) . Therefore, it is reasonable to expect that (22) could be relaxed to range ([ W 00 W 01 ]) ⊆ range ([ B 0 W 00 B 0 · · · ( W n − 1 ) 00 B 0 ]) . (30) Howe ver , it turns out that this relaxation is not possible, the reason being the (apparently simple, yet intricate) nonlinearity in (3). W e show this by means of an example. Example V .7. (Tightness of (22) ). Consider the feedback dynamics (3), (20), (26), where n = 3 , p = 1 , r = 2 , and W =   2 α 0 0 0 3 α 0 0 0 α   , B =   1 1 0   , α ∈ (0 . 5 , 1) . Clearly , (22) does not hold (so Theorem V .3(iv) does not apply), but range ([ W 00 W 01 ]) ⊆ range ([ B 0 W 00 B 0 ]) . One can show that for all K ∈ R 1 × 3 , ρ ( | W + BK | ) ≥ 2 α > 1 , while ρ ( W 11 ) = α < 1 , verifying that (22) is necessary and cannot be relaxed to (30).  Theorems V .2 and V .3 use completely dif ferent mechanisms for inhibition of N 0 2 , yet they are strikingly similar in one conclusion: that the dynamical properties achie vable under selectiv e inhibition are precisely those satisﬁed by N 1 2 . This has important implications for the size and structure of the part N 1 2 that can be activ e at any instance of time without resulting in instability . The next remark elaborates on this implication. Remark V .8. (Implications for the size and connection strength of N 1 2 ). Existing experimental e vidence suggest that the synaptic weights W in cortical networks are sparse, approximately follow a log-normal distrib ution, and ha ve a pairwise connection probability that is independent of physical distance between neurons within short distances [84]. Based on simulations of matrices with such statistics, Figure 5(a, b) show how quickly the network (representing N 2 here) mov es tow ards instability when its size grows. On the other hand, it is well-known that increasing n (and thus the number of synaptic weights) increases network expressivity (i.e., capacity to reproduce complex trajectories). While determining the optimal size of a network that leads to the best tradeoff between stability and expressi vity is beyond the scope of this work, our results suggest a critical role for selecti ve inhibition in keeping only a limited number of nodes in N 2 activ e at any giv en time while inhibiting others. In other words, while the ov erall size of subnetworks in a brain network (corresponding to, e.g., the number of neuronal populations with distinct preferred stimuli in a brain region) is inevitably lar ge, selective inhibition offers a plausible e xplanation for the mechanism by which the brain keeps the number of active populations bounded ( O (1) ) at any gi ven time. Similarly , Figure 5(c, d) show the transition of networks tow ards instability as their synaptic connections become stronger . While excitatory synapses, as expected, have a larger impact on stability , the same trend is also observed while vary- ing inhibitory synaptic strengths. Interestingly , several works in the neuroscience literature have shown that neuronal net- works maintain stability by re-scaling their synaptic weights that change during learning, a process commonly referred to as homeostatic synaptic plasticity [85]. Our results thus open the way to provide rigorous and quantiﬁable measures of the optimal size and weight distribution of subnetworks that may be acti ve at any given time and the homeostatic mechanisms that maintain an y desired le vel of stability and expressivity .  V I . C O N C L U S I O N S W e adopt a control-theoretic frame work, termed hierarchical selectiv e recruitment (HSR), as a mechanism to explain goal- driv en selecti ve attention. Moti v ated by the organization of the brain, HSR employs a hierarchical model which consists of an arbitrary number of neuronal subnetworks that operate at different layers of a hierarchy . While HSR is not conﬁned to any family of models, we here use the well-studied linear- threshold rate models to describe the dynamics at each layer of the hierarchy . W e provide a thorough analysis of the internal dynamics of each layer . Leveraging the switched-afﬁne nature of linear -threshold dynamics, we derive sev eral necessary and sufﬁcient conditions for the existence and uniqueness of equilibria (corresponding to P-matrices), local and global asymptotic stability (corresponding to totally-Hurwitz ma- trices), and boundedness of trajectories (corresponding to stability of excitatory-only dynamics). These results set the basis for analyzing the problem of selectiv e inhibition. W e show that using either feedforward or feedback inhibition, the dynamical properties of each layer after inhibition are precisely determined by the task-rele vant part that remains activ e. W e ha ve also provided constructiv e control designs that guarantee selectiv e inhibition under both schemes. Among the directions for future research, we highlight the study of output-feedback selective inhibition and the analysis of the conditions on (single-layer) linear-threshold networks that lead to the emergence of limit cycles and their stability . A C K N O W L E D G M E N T S W e would like to thank Dr . Erik J. Peterson for piquing our interest with his questions on dimensionality control in brain networks and for introducing us to linear-threshold modeling in neuroscience. W e are further indebted to the anonymous revie wer for suggesting the proof of the necessity of I − W ∈ P for the EUE of unbounded networks. This work was supported by NSF A ward CMMI-1826065 (EN and JC) and AR O A ward W911NF-18-1-0213 (JC). R E F E R E N C E S [1] E. Nozari and J. Cort ´ es, “Stability analysis of complex networks with linear -threshold rate dynamics, ” in American Contr ol Conference , Milwaukee, WI, May 2018, pp. 191–196. [2] J. Sully , “The psycho-physical process in attention, ” Brain , v ol. 13, no. 2, pp. 145–164, 1890. [3] E. C. Cherry , “Some experiments on the recognition of speech, with one and with two ears, ” The Journal of the Acoustical Society of America , vol. 25, no. 5, pp. 975–979, 1953. [4] R. Desimone and J. Duncan, “Neural mechanisms of selective visual attention, ” Annual Review of Neur oscience , v ol. 18, no. 1, pp. 193–222, 1995. [5] L. Itti and C. K och, “Computational modelling of visual attention, ” Natur e Revie ws Neuroscience , vol. 2, no. 3, p. 194, 2001. [6] N. La vie, A. Hirst, J. W . DeFockert, and E. V iding, “Load theory of selective attention and cognitive control. ” Journal of Experimental Psychology: General , vol. 133, no. 3, p. 339, 2004. [7] A. Gazzaley and A. C. Nobre, “T op-down modulation: bridging selecti ve attention and working memory , ” T rends in Cognitive Sciences , vol. 16, no. 2, pp. 129–135, 2012. [8] J. T . Serences and S. Kastner, “ A multi-lev el account of selective attention, ” The Oxfor d Handbook of Attention , p. 76, 2014. [9] D. E. Broadbent, Ed., P er ception and communication . Pergamon, 1958. [10] A. M. T reisman, “Strategies and models of selective attention. ” Psycho- logical re view , vol. 76, no. 3, p. 282, 1969. [11] J. Moran and R. Desimone, “Selective attention gates visual processing in the extrastriate cortex, ” Science , vol. 229, no. 4715, pp. 782–784, 1985. [12] B. C. Motter, “Focal attention produces spatially selective processing in visual cortical areas V1, V2, and V4 in the presence of competing stimuli, ” Journal of Neur ophysiology , vol. 70, no. 3, pp. 909–919, 1993. [13] S. Kastner , P . DeW eerd, R. Desimone, and L. G. Ungerleider, “Mecha- nisms of directed attention in the human e xtrastriate cortex as re vealed by functional MRI, ” Science , vol. 282, no. 5386, pp. 108–111, 1998. [14] M. A. Pinsk, G. M. Doniger , and S. Kastner , “Push-pull mechanism of selectiv e attention in human extrastriate cortex, ” J ournal of Neur ophys- iology , vol. 92, no. 1, pp. 622–629, 2004. [15] N. Lavie, “Distracted and confused?: Selective attention under load, ” T rends in Cognitive Sciences , vol. 9, no. 2, pp. 75–82, 2005. [16] J. J. Foxe and A. C. Snyder , “The role of alpha-band brain oscillations as a sensory suppression mechanism during selective attention, ” F rontier s in Psychology , vol. 2, p. 154, 2011. [17] H. Pashler , Attention . Psychology Press, 2016. [18] M. Gomez-Ramirez, K. Hysaj, and E. Nieb ur, “Neural mechanisms of selectiv e attention in the somatosensory system, ” Journal of neurophys- iology , vol. 116, no. 3, pp. 1218–1231, 2016. [19] E. Nozari and J. Cort ´ es, “Hierarchical selecti ve recruitment in linear- threshold brain networks. Part II: Inter-layer dynamics and top-down recruitment, ” IEEE T ransactions on A utomatic Control , vol. 66, 2021, to appear . A vailable at https://arxiv .org/abs/1809.02493. [20] F . Ratliff and H. K. Hartline, Studies on Excitation and Inhibition in the Retina . Rockefeller University Press, 1974. [21] K. P . Hadeler, “On the theory of lateral inhibition, ” K ybernetik , vol. 14, no. 3, pp. 161–165, 1973. (a) (b) (c) (d) Fig. 5: The ef fects of network size and weight distribution on its stability . In all panels, unless otherwise stated, W matrices are generated randomly with log-normally distrib uted entries with parameters µ = − 0 . 7 and σ = 0 . 9 as given in [84], 20% sparsity , and 80% excitatory nodes. Probabilities are estimated empirically with 10 3 samples and error bars represent standard error of the mean (s.e.m). (a) Probability of I − W ∈ P (as related to EUE, cf. Theorem IV .1) and − I + W ∈ H (as related to asymptotic stability , cf. Theorem IV .8) showing a rapid decay with n . (b) For large n , while checking I − W ∈ P and − I + W ∈ H can become prohibitive, their suf ﬁcient condition ρ ( | W | ) < 1 can be checked efﬁciently . The line illustrates a ﬁt of the form log ρ ( | W | ) = α log n + β with α = 1 and β = − 1 . 2 , sho wing a linear growth of ρ ( | W | ) with n . (c, d) The probabilities of − I + W ∈ H and ρ ( | W | ) < 1 while varying the µ parameter of the log-normal weight distribution ov er [ − 3 . 5 , 3 . 5] for both excitatory and inhibitory synapses (blue), only for excitatory ones (red), or only for inhibitory ones (yellow). The plots are very similar for I − W ∈ P and thus not sho wn. n = 10 is ﬁxed in these panels. [22] R. H. R. Hahnloser , H. S. Seung, and J. J. Slotine, “Permitted and forbid- den sets in symmetric threshold-linear networks, ” Neural Computation , vol. 15, no. 3, pp. 621–638, 2003. [23] Z. Y i, L. Zhang, J. Y u, and K. K. T an, “Permitted and forbidden sets in discrete-time linear threshold recurrent neural networks, ” IEEE T ransactions on Neural Networks , vol. 20, no. 6, pp. 952–963, 2009. [24] K. P . Hadeler and D. Kuhn, “Stationary states of the Hartline-Ratliff model, ” Biological Cybernetics , vol. 56, no. 5-6, pp. 411–417, 1987. [25] J. Feng and K. P . Hadeler, “Qualitativ e behaviour of some simple networks, ” Journal of Physics A: Mathematical and General , vol. 29, no. 16, pp. 5019–5033, 1996. [26] M. Forti and A. T esi, “New conditions for global stability of neural net- works with application to linear and quadratic programming problems, ” IEEE T ransactions on Circuits and Systems I: Fundamental Theory and Applications , vol. 42, no. 7, pp. 354–366, 1995. [27] M. Forti and P . Nistri, “Global con vergence of neural networks with discontinuous neuron activations, ” IEEE T ransactions on Circuits and Systems I: Fundamental Theory and Applications , v ol. 50, no. 11, pp. 1421–1435, 2003. [28] I. W . Sandberg and A. N. W illson Jr ., “Some theorems on properties of DC equations of nonlinear networks, ” Bell System T echnical Journal , vol. 48, no. 1, pp. 1–34, 1969. [29] H. Zhang, Z. W ang, and D. Liu, “ A comprehensiv e re view of stability analysis of continuous-time recurrent neural networks, ” IEEE T ransac- tions on Neural Networks and Learning Systems , vol. 25, no. 7, pp. 1229–1262, 2014. [30] Z. Yi and K. K. T an, “Multistability of discrete-time recurrent neural networks with unsaturating piecewise linear activ ation functions, ” IEEE T ransactions on Neural Networks , vol. 15, no. 2, pp. 329–336, 2004. [31] W . Zhou and J. M. Zurada, “ A new stability condition for discrete time linear threshold recurrent neural networks, ” in F ifth Int. Conf. on Intellig . Contr ol and Inf. Pr oc. , Aug 2014, pp. 96–99. [32] T . Shen and I. R. Petersen, “Linear threshold discrete-time recurrent neu- ral networks: Stability and globally attracti ve sets, ” IEEE T ransactions on Automatic Control , vol. 61, no. 9, pp. 2650–2656, 2016. [33] K. C. T an, H. T ang, and W . Zhang, “Qualitativ e analysis for recurrent neural networks with linear threshold transfer functions, ” IEEE T rans- actions on Cir cuits and Systems I: Re gular P apers , vol. 52, no. 5, pp. 1003–1012, 2005. [34] H. W ersing, W . Beyn, and H. Ritter , “Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions, ” Neural Computation , v ol. 13, no. 8, pp. 1811–1825, 2001. [35] K. Morrison, A. Degeratu, V . Itskov , and C. Curto, “Di versity of emer- gent dynamics in competitiv e threshold-linear networks: a preliminary report, ” arXiv pr eprint arXiv:1605.04463 , 2016. [36] H. Lin and P . J. Antsaklis, “Stability and stabilizability of switched linear systems: A survey of recent results, ” IEEE Tr ansactions on A utomatic Contr ol , vol. 54, no. 2, pp. 308–322, 2009. [37] D. Liberzon, Switching in Systems and Contr ol , ser . Systems & Control: Foundations & Applications. Birkh ¨ auser , 2003. [38] M. K. J. Johansson, Piecewise Linear Contr ol Systems: A Computational Appr oach , ser. Lecture Notes in Control and Information Sciences. Springer Berlin Heidelber g, 2003. [39] K. N. Seidl, M. V . Peelen, and S. Kastner , “Neural evidence for distracter suppression during visual search in real-w orld scenes, ” Journal of Neuroscience , vol. 32, no. 34, pp. 11 812–11 819, 2012. [40] S. Kastner, P . DeW eerd, M. A. Pinsk, M. I. Elizondo, R. Desimone, and L. G. Ungerleider, “Modulation of sensory suppression: implica- tions for recepti ve ﬁeld sizes in the human visual cortex, ” Journal of Neur ophysiology , v ol. 86, no. 3, pp. 1398–1411, 2001. [41] G. Rees, C. D. Frith, and N. Lavie, “Modulating irrelev ant motion perception by varying attentional load in an unrelated task, ” Science , vol. 278, no. 5343, pp. 1616–1619, 1997. [42] D. H. O’Connor, M. M. Fukui, M. A. Pinsk, and S. Kastner , “ Attention modulates responses in the human lateral geniculate nucleus, ” Natur e Neur oscience , vol. 5, no. 11, p. 1203, 2002. [43] M. Fiedler and V . Ptak, “On matrices with non-positiv e off-diagonal elements and positive principal minors, ” Czechoslovak Mathematical Journal , vol. 12, no. 3, pp. 382–400, 1962. [44] O. Slyusarev a and M. Tsatsomeros, “Mapping and preserver properties of the principal pi vot transform, ” Linear and Multilinear Algebr a , vol. 56, no. 3, pp. 279–292, 2008. [45] D. S. Bernstein, Matrix Mathematics , 2nd ed. Princeton University Press, 2009. [46] P . Dayan and L. F . Abbott, Theoretical Neur oscience: Computational and Mathematical Modeling of Neural Systems , ser . Computational Neuroscience. Cambridge, MA: MIT Press, 2001. [47] D. A. Henze, Z. Borhe gyi, J. Csicsvari, M. A. Mamiya, K. D. Harris, and G. Buzsaki, “Intracellular features predicted by extracellular recordings in the hippocampus in vivo, ” Journal of Neur ophysiology , vol. 84, no. 1, pp. 390–400, 2000. [48] D. A. Henze, K. D. Harris, Z. Borhegyi, J. Csicsvari, A. Mamiya, H. Hirase, A. Sirota, and G. Buzsaki, “Simultaneous intracellular and extracellular recordings from hippocampus region ca1 of anesthetized rats, ” CRCNS.org, 2009. [Online]. A vailable: http: //dx.doi.org/10.6080/K02Z13FP [49] Y . Ahmadian, D. B. Rubin, and K. D. Miller , “ Analysis of the stabilized supralinear network, ” Neural Computation , vol. 25, no. 8, pp. 1994– 2037, 2013. [50] H. K. Khalil, Nonlinear Systems , 3rd ed. Prentice Hall, 2002. [51] R. T . Rockafellar and R. J. B. W ets, V ariational Analysis , ser . Com- prehensiv e Studies in Mathematics. Ne w Y ork: Springer, 1998, vol. 317. [52] L. E. J. Brouwer , “ ¨ Uber abbildung von mannigfaltigk eiten, ” Mathema- tische Annalen , vol. 71, no. 1, pp. 97–115, 1911. [53] D. Kuhn and R. L ¨ owen, “Piecewise afﬁne bijections of R n , and the equation S x + − T x − = y , ” Linear Algebra and its Applications , vol. 96, pp. 109–129, 1987. [54] K. G. Murty , “On the number of solutions to the complementarity problem and spanning properties of complementary cones, ” Linear Algebra and its Applications , vol. 5, no. 1, pp. 65–108, 1972. [55] D. M. Stipanovic and D. D. Siljak, “Stability of polytopic systems via conv ex M-matrices and parameter-dependent L yapunov functions, ” Nonlinear Analysis, Theory , Methods & Applications , v ol. 40, no. 1, pp. 589–609, 2000. [56] G. E. Coxson, “The P-matrix problem is co-NP-complete, ” Mathemati- cal Pro gramming , vol. 64, no. 1, pp. 173–178, 1994. [57] S. M. Rump, “On P-matrices, ” Linear Algebra and its Applications , vol. 363, pp. 237–250, 2003. [58] H. R. Wilson and J. D. Cowan, “Excitatory and inhibitory interactions in localized populations of model neurons, ” Biophysical Journal , vol. 12, no. 1, pp. 1–24, 1972. [59] A. C. E. Onslow , M. W . Jones, and R. Bogacz, “ A canonical circuit for generating phase-amplitude coupling, ” PLOS One , vol. 9, no. 8, p. e102591, 2014. [60] M. P . Jadi and T . J. Sejnowski, “Regulating cortical oscillations in an inhibition-stabilized network, ” Proceedings of the IEEE , vol. 102, no. 5, pp. 830–842, 2014. [61] J. J. Hopﬁeld, “Neural networks and physical systems with emergent collectiv e computational abilities, ” Proceedings of the National Academy of Sciences , v ol. 79, no. 8, pp. 2554–2558, 1982. [62] H. S. Seung, D. D. Lee, B. Y . Reis, and D. W . T ank, “Stability of the memory of eye position in a recurrent network of conductance-based model neurons, ” Neur on , vol. 26, no. 1, pp. 259–271, 2000. [63] D. Durste witz, J. K. Seamans, and T . J. Sejnowski, “Neurocomputational models of working memory , ” Natur e Neur oscience , vol. 3, no. 11s, p. 1184, 2000. [64] R. Cossart, D. Aronov , and R. Y uste, “ Attractor dynamics of network up states in the neocortex, ” Natur e , vol. 423, no. 6937, pp. 283–288, 2003. [65] J. J. Knierim and K. Zhang, “ Attractor dynamics of spatially correlated neural activity in the limbic system, ” Annual Revie w of Neuroscience , vol. 35, no. 1, pp. 267–285, 2012. [66] A. Pa vlov , N. van de W ouw , and H. Nijmeijer , “Con vergent piece wise afﬁne systems: analysis and design Part I: continuous case, ” in IEEE Conf. on Decision and Contr ol , Dec 2005, pp. 5391–5396. [67] E. Nozari and J. Cort ´ es, “Hierarchical selecti ve recruitment in linear- threshold brain networks. Part I: Intra-layer dynamics and selectiv e inhibition, ” Pr eliminary version, available online at https:// arxiv .org/ abs/ 1809.01674v2 , 2019. [68] D. Liberzon and R. T empo, “Common Lyapunov functions and gradient algorithms, ” IEEE T ransactions on Automatic Control , vol. 49, no. 6, pp. 990–994, 2004. [69] E. Nozari and J. Cort ´ es, “Oscillations and coupling in interconnections of two-dimensional brain networks, ” in American Contr ol Confer ence , Philadelphia, P A, July 2019, pp. 193–198. [70] Y . Qin, Y . Kawano, and M. Cao, “Partial phase cohesiveness in net- works of communitinized Kuramoto oscillators, ” in European Control Confer ence , 2018, pp. 2028–2033. [71] M. Jafarian, X. Y i, M. Pirani, H. Sandber g, and K. Henrik Johansson, “Synchronization of kuramoto oscillators in a bidirectional frequency- dependent tree network, ” in IEEE Conf. on Decision and Contr ol , 2018, pp. 4505–4510. [72] T . Menara, G. Baggio, D. S. Bassett, and F . Pasqualetti, “Stability condi- tions for cluster synchronization in netw orks of heterogeneous K uramoto oscillators, ” IEEE T ransactions on Contr ol of Network Systems , 2019, in press. [73] H. G. Schuster and P . W agner, “ A model for neuronal oscillations in the visual cortex. 1. mean-ﬁeld theory and deriv ation of the phase equations, ” Biological Cybernetics , vol. 64, no. 1, pp. 77–82, 1990. [74] P . Jiruska, J. Csicsvari, A. D. Po well, J. E. Fox, W . Chang, M. Vreugden- hil, X. Li, M. P alus, A. F . Bujan, R. W . Dearden, and J. G. R. Jef ferys, “High-frequency network activity , global increase in neuronal activity , and synchrony expansion precede epileptic seizures in vitro, ” J ournal of Neur oscience , vol. 30, no. 16, pp. 5690–5701, 2010. [75] S. K. Y . Nikrav esh, Nonlinear Systems Stability Analysis: Lyapuno v- Based Approac h . CRC Press, 2013. [76] J. S. Isaacson and M. Scanziani, “How inhibition shapes cortical activity , ” Neuron , vol. 72, no. 2, pp. 231–243, 2011. [77] E. D. Sontag, “On the input-to-state stability property , ” European Journal of Contr ol , vol. 1, pp. 24–36, 1995. [78] C. R. Johnson and H. A. Robinson, “Eigen value inequalities for principal submatrices, ” Linear Algebra and its Applications , vol. 37, pp. 11–22, 1981. [79] R. A. Nicoll, “ A brief history of long-term potentiation, ” Neuron , vol. 93, no. 2, pp. 281–290, 2017. [80] R. T remblay , S. Lee, and B. Rudy , “Gabaergic interneurons in the neocortex: from cellular properties to circuits, ” Neuron , vol. 91, no. 2, pp. 260–292, 2016. [81] P . Somogyi, G. T amasab, R. Lujan, and E. H. Buhl, “Salient features of synaptic organisation in the cerebral cortex, ” Brain Resear ch Reviews , vol. 26, no. 2, pp. 113–135, 1998. [82] H. L. Royden and P . Fitzpatrick, Real Analysis . Prentice Hall, 2010. [83] C. T . Chen, Linear System Theory and Design , 3rd ed. New Y ork, NY , USA: Oxford Uni versity Press, Inc., 1998. [84] S. Song, P . J. Sj ¨ ostr ¨ om, M. Reigl, S. Nelson, and D. B. Chklovskii, “Highly nonrandom features of synaptic connectivity in local cortical circuits, ” PLOS Biology , vol. 3, no. 3, p. e68, 2005. [85] G. T urrigiano, “Homeostatic synaptic plasticity: local and global mech- anisms for stabilizing neuronal function, ” Cold Spring Harbor P erspec- tives in Biology , vol. 4, p. a005736, 2012. A P P E N D I X A . A D D I T I O NA L L E M M A S A N D P R O O F S Pr oof of Theor em IV .6: The necessity is tri vial since M − 1 0 M ` = − ( I − W ) . T o prove sufﬁciency , note that for an y σ ∈ { 0 , ` } n , M − 1 σ = ( I − WΣ ` )(2 Σ ` − I ) = (2 Σ ` − I ) − WΣ ` . (31) Since nodes can be relabeled arbitrarily , we can assume without loss of generality that σ 1 = [ ` T n 1 ` T n 2 0 T n 3 0 T n 4 ] T and σ 2 = [ ` T n 1 0 T n 2 ` T n 3 0 T n 4 ] T where n 1 , . . . , n 4 ≥ 0 , P 4 i =1 n i = n . Then, it follows from (31) that M − 1 σ 1 =     I n 1 − W 11 − W 12 0 0 − W 21 I n 2 − W 22 0 0 − W 31 − W 32 − I n 3 0 − W 41 − W 42 0 − I n 4     , M − 1 σ 2 =     I n 1 − W 11 0 − W 13 0 − W 21 − I n 2 − W 23 0 − W 31 0 I n 3 − W 33 0 − W 41 0 − W 43 − I n 4     , where W ij ’ s are submatrices of W with appropriate dimen- sions. T aking the in verse of M − 1 σ 1 as a 2 -by- 2 block-triangular matrix [45, Prop 2.8.7] (with the indicated blocks), we get M σ 1 =      I n 1 − W 11 − W 12 − W 21 I n 2 − W 22  − 1 0 −  W 31 W 32 W 41 W 42   I n 1 − W 11 − W 12 − W 21 I n 2 − W 22  − 1 − I n 3 + n 4     , so direct multiplication giv es M σ 1 M − 1 σ 2 =  B 1 B 2 B 3 B 4  , with B 1 =  I n 1 − W 11 − W 12 − W 21 I n 2 − W 22  − 1  I n 1 − W 11 0 − W 21 − I n 2  , B 2 = −  I n 1 − W 11 − W 12 − W 21 I n 2 − W 22  − 1  W 13 0 W 23 0  , B 3 = −  W 31 W 32 W 41 W 42  B 1 +  W 31 0 W 41 0  , B 4 = −  W 31 W 32 W 41 W 42  B 2 −  I n 3 − W 33 0 − W 43 − I n 4  . W ith this, after some computations one can show that M σ 1 M − 1 σ 2 =     I n 1 ? ? 0 0 0  ? ? ? ?  0 0 0 ? ? I n 4     . (32) Let Γ ∈ R ( n 2 + n 3 ) × ( n 2 + n 3 ) be the bracketed block in M σ 1 M − 1 σ 2 and deﬁne Q =  Q 11 Q 12 Q 21 Q 22  ,  I n 1 − W 11 − W 12 − W 21 I n 2 − W 22  − 1 , R , I n 3 − W 33 −  W 31 W 32  Q  W 13 W 23  . It can be shown that Γ = −     Q 22  Q 21 Q 22   W 13 W 23  −  W 31 W 32   Q 12 Q 22  R     . In verting the left-hand-side matrix (below) as a 2-by-2 block matrix [45, Prop 2.8.7] (the ﬁrst block is Q − 1 ) and applying the matrix in version lemma [45, Cor 2.8.8] to the ﬁrst block of the result, we obtain   I n 1 − W 11 − W 12 − W 13 − W 21 I n 2 − W 22 − W 23 − W 31 − W 32 I n 3 − W 33   − 1 =   ? ? ? ? ˆ B 1 ˆ B 2 ? ˆ B 3 ˆ B 4   where ˆ B 1 = Q 22 +  Q 21 Q 22   W 13 W 23  R − 1  W 31 W 32   Q 12 Q 22  , ˆ B 2 =  Q 21 Q 22   W 13 W 23  R − 1 , ˆ B 3 = R − 1  W 31 W 32   Q 12 Q 22  , and ˆ B 4 = R − 1 . Therefore, − Γ is the principal piv ot transform of  ˆ B 1 ˆ B 2 ˆ B 3 ˆ B 4  so if I − W ∈ P , Lemma II.2(v) and the block structure of (32) guarantee that − M σ 1 M − 1 σ 2 ∈ P . The following result is used in the proof of Theorem V .2. Lemma A.1. (GES of cascaded interconnections). Consider the cascaded dynamics τ ˙ x 0 = − x 0 , τ ˙ x 1 = − x 1 + [ W 10 x 0 + W 11 x 1 + ˜ d 1 ] m 1 0 , (33) wher e x 0 ∈ R r and x 1 ∈ R n − r . If W 11 is such that τ ˙ x 1 = − x 1 + [ W 11 x 1 + ˜ d 1 ] m 1 0 , (34) is GES for any constant ˜ d 1 ∈ R n − r , then the whole dynam- ics (33) is also GES for any constant ˜ d 1 . Pr oof: W e only prove the result for ˜ d 1 = 0 . This is without loss of generality , since for ˜ d 1 6 = 0 , we can apply the change of variables ξ = x − x ∗ , where x ∗ is the equilibrium corresponding to input [ 0 T ( ˜ d 1 ) T ] T , and shift the equilibrium to the origin. Since (34) is GES, [19, Thm A.1] guarantees that there exists x 1 7→ V 1 ( x 1 ) such that c 1 k x 1 k 2 ≤ V 1 ( x 1 ) ≤ c 2 k x 1 k 2 , (35a)    ∂ V 1 ( x 1 ) ∂ x 1    ≤ c 3 k x 1 k , (35b) for some c 1 , c 2 , c 3 > 0 , and, if x 1 ( t ) is the solution of (34), τ d dt V 1 ( x 1 ( t )) ≤ − c 4 k x 1 k 2 , (35c) for some c 4 > 0 . Since [ · ] m 1 0 is Lipschitz continuous, it follo ws from (35b) and (35c) that if x 1 ( t ) is the solution of (33), τ d dt V 1 ( x 1 ( t )) ≤ − c 4 k x 1 k 2 + c 3 k x 1 kk W 10 x 0 k ≤ − c 4 2 k x 1 k 2 + c 2 3 k W 10 k 2 2 c 4 k x 0 k 2 , where the second inequality follows from Y oung’ s inequal- ity [50, p. 466]. No w , let V ( x ) = ( c 2 3 k W 10 k 2 / 2 c 4 ) k x 0 k 2 + V 1 ( x 1 ) . It is straightforward to verify that V satisﬁes all the assumptions of [50, Thm 4.10] with a = 2 . Erfan Nozari received his B.Sc. degree in Electrical Engineering-Control in 2013 from Isfahan Univer - sity of T echnology , Iran and Ph.D. in Mechanical Engineering and Cognitiv e Science in 2019 from Univ ersity of California San Die go. He is currently a postdoctoral researcher at the University of Penn- sylvania Department of Electrical and Systems En- gineering. He has been the (co)recipient of the 2019 IEEE T ransactions on Control of Network Systems Outstanding Paper A ward, the Best Student Paper A ward from the 57th IEEE Conference on Decision and Control, the Best Student Paper A ward from the 2018 American Control Conference, and the Mechanical and Aerospace Engineering Distinguished Fellowship A ward from the Univ ersity of California San Die go. His research interests include dynamical systems and control theory and its applications in computational and theoretical neuroscience and complex network systems. Jorge Cort ´ es (M’02-SM’06-F’14) recei ved the Li- cenciatura de gree in mathematics from Universidad de Zaragoza, Zaragoza, Spain, in 1997, and the Ph.D. degree in engineering mathematics from Uni- versidad Carlos III de Madrid, Madrid, Spain, in 2001. He held postdoctoral positions with the Uni- versity of T wente, T wente, The Netherlands, and the Univ ersity of Illinois at Urbana-Champaign, Urbana, IL, USA. He was an Assistant Professor with the Department of Applied Mathematics and Statistics, Univ ersity of California, Santa Cruz, CA, USA, from 2004 to 2007. He is currently a Professor in the Department of Mechanical and Aerospace Engineering, University of California, San Diego, CA, USA. He is the author of Geometric, Control and Numerical Aspects of Nonholonomic Systems (Springer-V erlag, 2002) and co-author (together with F . Bullo and S. Mart ´ ınez) of Distributed Control of Robotic Networks (Princeton University Press, 2009). He is a Fellow of IEEE and SIAM. His current research interests include distributed control and optimization, network science, opportunistic state-triggered control, reasoning and decision making under uncertainty , and distributed coordination in po wer networks, robotics, and transportation.

Hierarchical Selective Recruitment in Linear-Threshold Brain Networks, Part I: Single-Layer Dynamics and Selective Inhibition

Original Paper

Comments & Academic Discussion

Leave a Comment

Original Paper

Related Papers

Comments & Academic Discussion

Leave a Comment