A Sherman-Morrison-Woodbury Identity for Rank Augmenting Matrices with Application to Centering

A SHERMAN MORR ISON W OODBUR Y IDENTITY F OR RANK A UGMENTING MA TRICES WITH APPLICA TION TO CENTERING ∗ KUR T S. R IEDEL † SIAM J. M A T. A NAL. c  1991 So ciety for Industrial and Applied Mathematics V ol. 12, No. 1, pp. 80–95 , January 199 1 000 Abstract. Matrices of the form A + ( V 1 + W 1 ) G ( V 2 + W 2 ) ∗ are considered where A is a sing ular ℓ × ℓ matrix and G i s a nonsingular k × k matrix, k ≤ ℓ . Let the columns of V 1 be i n the column s pace of A an d the columns of W 1 be orthogonal to A . Sim ilarly , let the columns of V 2 be in the column space of A ∗ and the columns of W 2 be orthogonal to A ∗ . An explicit expression for the inv erse is giv en, provided that W ∗ i W i has r ank k . An ap pl ication to centering cov ariance matrices about t he mean is given. Key words. Li near Algebra, Sch ur Matrices, Generalized Inv erses AMS(MOS) sub ject classi fica tions. 65 R10, 33A65, 35K05, 62G 20, 65P05 The wellkno wn Sherman-Morr ison-W o o dbury matrix iden tity [1]: ( A + X 1 G X T 2 ) − 1 = A − 1 − A − 1 X 1 ( G − 1 + X T 2 A − 1 X 1 ) − 1 X T 2 A − 1 (1) is widely used. Sev era l excellent review articles hav e appea r ed r ecently [2-4]. How ever (1) is only v alid when A is nonsing ular 1 . In this article, we consider matrix inv er ses of the form A + X 1 G X T 2 where the rank of A + X 1 G X T 2 is lar ger than t he rank of A . W e decomp os e the matrix X 1 int o V 1 + W 1 , where the columns of V 1 are contained in the column space of A and the columns of W 1 are ortho g onal to it. Similarly , w e decomp ose X 2 int o V 2 + W 2 , where the columns of V 2 are con tained in t he column space of A ∗ and the columns of W 2 are orthogonal to M ( A ∗ ) . W e denote the column spa ce o f A by M ( A ). The Mo ore-Penrose genera lized inv erse will b e denoted b y the sup erscr ipt + . W e deno te the k × k matrix W ∗ i W i by B i and define C i ≡ W i ( W ∗ i W i ) − 1 . W e will r equire B i to b e nonsingular. Ho wever the ra nk of the p erturbation, k , can be significan tly less than the size of the original matrix. W e note t ha t V ∗ i W i = 0 and W ∗ i C i = I k . Finally the pr o jection oper ator onto the column spa ce of W satisfies W i B − 1 i W ∗ i = W 1 C ∗ 1 = C 2 W ∗ 2 . Theorem 1 . Let A b e a ℓ × ℓ ma trix o f r ank ℓ 1 , ℓ 1 < ℓ , V i and W i be ℓ × k matrices and G b e a k × k nonsingular matrix . Let the columns of V 1 ∈ M ( A ) and the columns of W 1 be or thogonal to M ( A ). Similarly , let the columns of V 2 ∈ M ( A ∗ ) and the columns of W 2 be or thogonal to M ( A ∗ ). Let B i ≡ W ∗ i W i hav e rank k . The matrix, Ω ≡ A + ( V 1 + W 1 ) G ( V 2 + W 2 ) ∗ , has the following Moore -Penrose genera lized in verse: Ω + = A + − C 2 V ∗ 2 A + − A + V 1 C ∗ 1 + C 2 ( G + + V ∗ 2 A + V 1 ) C ∗ 1 . (2) ∗ Receiv ed b y the editors July 12, 1990; accept ed for publication F ebruary 19, 1991. † Couran t Institute of Mathematical Sciences, New Y ork Universit y , New Y ork, New Y or k 10012. The work of this auth or w as sup p orted b y the U.S. Departmen t of Energy Grant No. DE- F G02- 86ER53223. 1 W e denote the transpose of a matrix, A by A T and the hermitian or conjugat e transp ose b y A ∗ . 79 80 Kur t S. Riedel Pro of: W e recall that the Mo o re Penrose inv er se is the unique generalized inv erse which satisfies the fo llowing fo ur conditions,(Ref. [5], p.26): ( a ) ΩΩ + Ω = Ω , ( b ) Ω + ΩΩ + = Ω + , ( c ) (ΩΩ + ) ∗ = Ω Ω + , ( d ) (Ω + Ω) ∗ = Ω + Ω . The identit y is verified b y direct computation, ΩΩ + ≡ A A + − A C 2 V ∗ 2 A + − A A + V 1 C ∗ 1 + A C 2 ( G + + V ∗ 2 A + V 1 ) C ∗ 1 +( V 1 + W 1 ) G ( V 2 + W 2 ) ∗ A + − ( V 1 + W 1 ) G ( V 2 + W 2 ) ∗ C 2 V ∗ 2 A + − ( V 1 + W 1 ) G ( V 2 + W 2 ) ∗ A + V 1 C ∗ 1 +( V 1 + W 1 ) G ( V 2 + W 2 ) ∗ C 2 ( V ∗ 2 A + V 1 ) C ∗ 1 +( V 1 + W 1 ) G ( V 2 + W 2 ) ∗ C 2 G + C ∗ 1 . Since W 2 is orthogona l to A ∗ , w e hav e A W 2 = 0 , W ∗ 2 A + = 0 , a nd V ∗ 2 W 2 = 0 , which simplifies the previous expression to ΩΩ + ≡ A A + − A A + V 1 C ∗ 1 + ( V 1 + W 1 ) G V ∗ 2 A + − ( V 1 + W 1 ) G W ∗ 2 C 2 V ∗ 2 A + − ( V 1 + W 1 ) G V ∗ 2 A + V 1 C ∗ 1 +( V 1 + W 1 ) G W ∗ 2 C 2 V ∗ 2 A + V 1 C ∗ 1 + ( V 1 + W 1 ) G W ∗ 2 C 2 G + C ∗ 1 . This express io n ma y b e simplified using G W ∗ 2 C 2 G + C ∗ 1 = C ∗ 1 , and G W ∗ 2 C 2 V ∗ 2 = G V ∗ 2 , and A A + V 1 = V 1 to ΩΩ + ≡ A A + + W 1 C ∗ 1 , and clearly condition (c) is satisfied. The corresp onding iden tit y for Ω + Ω ≡ A + A + C 2 W ∗ 2 requires the decomposi- tion to satisfy A + W 1 = 0 , W ∗ 1 A = 0 , V ∗ 1 W 1 = 0, a nd V 2 A + A = V 2 . In addi- tion, the matr ix G must satisfy C 2 G + C ∗ 1 W 1 G = C 2 and V 1 C ∗ 1 W 1 G = V 1 G . These requirements guarantee that conditions (a), (b) and (d) are als o satisfied. [] Remark: The conditions tha t G and W ∗ i W i hav e rank k may b e re pla ced b y the somewhat w eaker but more complicated co nditions that G W ∗ 2 C 2 G + C ∗ 1 = C ∗ 1 , G W ∗ 2 C 2 V ∗ 2 = G V ∗ 2 , C 2 G + C ∗ 1 W 1 G = C 2 and V 1 C ∗ 1 W 1 G = V 1 G . Note that the generalized in verse in (2) is singular and tends t o infinit y as W i approaches to zero. Thus (2) do es not reduce to the (1) a s the perturbatio n t ends to zero. When the perturba tio n of the column space of A is zero, i.e. V ≡ 0, theor em 1 simplifies to Ω + = A + + C 2 G + C 1 . (3) When A is a symmetric matrix, the column s paces of A and A ∗ are iden tical. Thu s , for the case of symmetric A and Ω, Thm. 1 reduces to Theorem 2 . Let A be a symmetric ℓ × ℓ matrix of rank ℓ 1 , ℓ 1 < ℓ , V and W be ℓ × k matrice s and G b e a k × k nonsingula r matrix. Let V ∈ M ( A ) and the Sherman M orrison Identity for Ran k Augmenting 81 columns of W b e or thogonal to M ( A ). Let B ≡ W ∗ W hav e rank k . The matrix , Ω ≡ A + ( V + W ) G ( V + W ) ∗ , has the following Moore -Penrose genera lized in verse: Ω + = A + − C V ∗ A + − A + V C ∗ + C ( G + + V ∗ A + V ) C ∗ . (4) F or concr eteness, we sp ecia lis e the preceding identities to the ca se o f rank one per turbations. In this sp ecial ca se, k ≡ 1, a nd V i and W i reduce to ℓ vectors, v i and w i . In the nonsingular case, (1) reduces to Bar tlett’s identit y [6]. It states for an arbitrar y nonsingular ℓ × ℓ matrix A and ℓ v ec tors v i , ( A + v 1 v 2 ∗ ) − 1 = A − 1 − ( A − 1 v 1 )( v 2 ∗ A − 1 ) (1 + v 2 ∗ A − 1 v 1 ) . (5) In t his case, theorem 1 reduces to the analogous r esult for an a rbitrary singular matrix A with a rank o ne pertur ba tion which con tains a comp onent p e rp endicular to the column space of A . Noting that G ≡ 1 and C i ≡ w i / | w i | 2 , theorem 1 simplifies to the following result. Theorem 3 . Let A be a ℓ × ℓ matrix of rank ℓ 1 , ℓ 1 < ℓ , and v i , w i , i = 1 , 2 be ℓ vectors. Let v 1 ∈ M ( A ) and w 1 be orthogonal to M ( A ), and v 2 ∈ M ( A ∗ ) and w 2 be orthogonal t o M ( A ∗ ). Assume w 2 is paralle l to w 1 and w i 6 = 0. Let Ω ≡ A + ( v 1 + w 1 )( v 2 + w 2 ) ∗ , The Mo ore-Penrose generalized in verse is Ω + = A + − w 2 v 2 ∗ A + | w 2 | 2 − A + v 1 w 1 ∗ | w 1 | 2 + (1 + v 2 ∗ A + v ) w 2 w 1 ∗ | w 1 | 2 | w 2 | 2 . (6) This generalized inv erse is singular and tends to infinity as 1 / | w 1 || w 2 | , as w i approaches to zero. Thus (6) does not r educe to Bartlett’s iden tity . The pro jection oper ator on to the r ow space of Ω is P X T = A + A + w i w i ∗ | w i | 2 . The symmetric version of Thm. 3 was originally developed and applied by the author in his s tatistical analysis of magnetic fusion data [7 ]. T o estimate the r egressio n parameters in ordinary least s quares reg ression, the sum of sq uares and pro ducts (SSP) matrix needs to be in verted. W e apply Thm. 3 to determine the in verse of the SSP matrix in terms of the inv ers e of the cov ar iance matrix of the cov aria tes. W e decompo se the indep endent v ariable vector, x int o a mean v a lue v ector , ¯ x and a fluctuating part, ˜ x . Thus the i -th individual obser v ation has the form x i = ¯ x + ˜ x i . Let X denote the n × ℓ data matrix whose rows consis t of x ∗ i and ˜ X be the centered data matrix whose rows consist of ˜ x ∗ i . W e assume that some of the indep endent v aria bles, x k , ha ve not b een v aried. Thu s ˜ X ∗ ˜ X is singula r. The inv erse of the uncentered sum of squares and cro sspro ducts 82 Kur t S. Riedel matrix, X ∗ X can now b e e xpressed in terms o f the Moo r e Penrose generaliz e d in verse of the centered co v arianc e matrix, ˜ X ∗ ˜ X . W e decomp ose a multiple of the mea n v alue vector, √ n ¯ x , in to v + w , where v ∈ M ( ˜ X ∗ ˜ X ) and w ⊥ M ( ˜ X ∗ ˜ X ). The data matrix has the form X ∗ X = ˜ X ∗ ˜ X + n ¯ x ¯ x T = ˜ X ∗ ˜ X + ( v + w )( v + w ) ∗ . Thu s w e hav e rewr itten X ∗ X in a f o rm appropr iate to the applica tion of theore m 3. In co nclus ion, the applica tion of these matrix identities requires the decomp osition of X i int o the or thogonal co mpo nent s, V i and W i . Th us our theorems are most useful in situations where the decomp ositio n is tr iv ial. A cknow le dgments The helpful comments o f the referees are gra tefully acknowledged. REFERENC ES 1. W.J. Duncan, “Some de v ices for the solution of lar ge s ets o f simultaneous equations (with an app endix on the recipro cation of partitioned matrices)”, The L ondon, Edinbur gh and D ublin Philosoph ic al M agazine and Journal of Scienc e , Seven th Series, 35 , p. 660, (1944). 2. H.V. Henderson and S.R. Searle, “O n deriving the in verse of a s um o f ma tri- ces”, SIAM Review, 23 , p.53, (198 1). 3. D.V. Ouellete, “Sch ur complemen ts and statistics”, J o urnal of Linear Alge- bra, 36 , p. 187, (1981). 4. W.W. Hager, “Up dating the inv ers e of a matr ix”, SIAM Review, 31 , p.221, (1989). 5. C.R. Rao, Line ar Statistic al In fer enc e and Its Applic ations , p. 26,33 , J. Wiley and Sons, New Y ork, 1973 . 6. M.S. Bartlett, “An in verse matrix adjustmen t ar is ing in discriminan t a naly- sis”, Annals of Ma t hematic al Statistics , vol. 2 2, p107, (1951). 7. K.S. Riedel, New Y ork Univ er s it y Re p or t MF-11 8, National T echnical In- formation Service do c ument no . DOE/ER/ 5 3223- 98 ”Adv anced Statis tics for T ok amak T ransp or t: Collinearity and T o k amak to T o k amak V ariation” (1989).