Reliability of Module Based Software System

Reliabilit y of Mo dule Based Soft w are System Rudrani Banerjee and Angsh um an Sark ar ∗ Dep artment of Statistics, Visva- Bhar ati University, India Octob er 30, 2018 Abstract This pap er consid er the problem of determining the reliabilit y of a softw are system whic h can b e decomp osed in a n umb er of mod ules. W e ha ve deriv ed the expression of the relia b ilit y of a syste m using the Mark o vian mo d el for the transfer of con trol b etw een mo du les in order. W e ha ve giv en th e expression of r eliabilit y b y considering b oth b enign and catastrophic failure. The expression of reliabilit y present ed in this work is applicable for some control soft ware whic h are d esigned to detect its o wn internal errors. 1 In tro duction No w a days larg e scale softw are systems are used in ev ery walk of life. The price of soft ware a re muc h higher than the cost of hardware wh en w e consider a huge computer in tensiv e system. Moreo v er the p enalt y cost incurred b y a false o utcome of a syste m is enormous. T o address suc h a c hallenge p osed b y this tec hnolog ical trend, during the last three decades extensiv e rese a rc h ha s fo cused on the a rea of softw are reliabilit y . The consideration of soft w a r e reliabilit y is increasing because of the growin g emph asis on soft ware that is reusable (as opp osed t o softw are that is written for a terminal mission), where it is essen tial to demonstrate that the system will p erform reliably for a v ariet y of end-user applic ations. A softw are system is defined here as a ” c o llection of programs and system files suc h that the system files are accessed and altered only by the programs in the collection ”. Eac h elemen t in this collection will b e called a mo dule - for instance, a mo dule migh t b e a progra m, a subprogr a m, or a file. The perfor ma nce ( and hence the reliability ) of the system clearly dep ends on that o f e ac h individual mo dule and the relationship betw een these mo dules and the system; in this regard a soft w a r e system is quite similar to an y other sys t em. How ev er, the actual relationship b etw een system and module reliabilities is quite unique a nd dep ends on the sp ecific definition of soft ware reliability as we ll a s on ∗ Email:sangs hu 20 00@yaho o.com 1 the structure of the ov erall system. In this paper w e focus o n softw are systems that c a n b e de comp osed in to a finite num b er of mo dules. In testing a soft ware one ma y t est the system a s a whole, but in practice, differen t organizational en tities are assigned responsibilit y of dev eloping differen t mo dules. So it will b e more beneficial in the contex t of b oth cost and time test the indiv idual mo dules instead of testing t hem tog ether. In order to do this, some mathematical mo dels, of- ten referred to as Softw are Reliabilit y Grow th Mo dels (SR G M) ar e used to enable the soft ware reliabilit y practitioners to estimate the exp ected future reliabilit y of a s o ft ware under dev elopmen t and accordingly allo cate time, money , hum an resources to a pro j ect. Often t hese reliabilit y gro wth mo dels express soft w are system reliability in terms of the individual module reliabilities w hich is fa v ora ble under b oth time and cost constrain ts. Cheung (2), first expressed the system reliability in terms of the comp onent relia- bilities. P o ore et al. (1) suggest allo cating the ta rgeted system reliability goal among the comp o nen ts and then testing the individual compo nen ts to v erify whether t he com- p onen t reliabilities meet the allo cated goa ls at a sp ecified lev el of confidence, where as Easterling, Mazum dar, Spencer and Diegert (6), has discussed this me tho d ma y lead to estimates of o v erly conserv ativ e sample size requiremen ts for comp onen t testing. Y ang et. al. has implemen ted the idea of using testabilit y to es timate softw are reliabilit y . They hav e a lso provide d the basic steps in v olv e estimating testabilit y , ev aluating how w ell softw are w as written, a nd assessing the relationship b etw een testing and usage b y assuming the mo dules a re indep enden tly f unctioning. They hav e also compared their results with those o btained b y using tw o reliability gro wth mo dels. Ra jgopal et. al. has used a Marko vian mo del for the transfer of con trol b et w een mo dules in order to dev elop the system r eliability expression in terms of the mo dule reliabilities in case of a dep enden t setup. T hey ha ve also discussed a pro cedure for determining the minim um n um b er of tests required of eac h module suc h that the pro babilit y of certifying a system whose reliabilit y falls b elo w a sp ecified v alue R 0 is less than a sp ecified small fraction β . Bonda v alli et. al. has considered the concept of b enign fa ilur e and catastrophic failure for determining the softw are reliabilit y for a iterative program. In this pap er we ha v e expres sed t he system reliabilit y in t erms of testabilit y of a particular mo dule f ollo wing Y a ng et. al. for dep enden t mo dules and hav e in t r o duced the concepts of benign and catastrophic failure follo wing Bonda v alli et. a l. in case of a system where it can b e decomp osed in a finite num b er o f dep enden tly functional mo d- ules. The section 2 discuss the notat io ns a nd preliminaries, section 3 giv es the expression of the probability of correct o utput for a sp ecific input. Recen t researc h [26] has sho wn a strong correlation b et we en reliability and co v erage criteria (Lott et al. (2005), K h un et. al. (20 02), Yilmaz et. al. (2004) etc.), although it is v ery difficult t o quan tify t his relation. Dalal et al. [6] and many more has examined this relatio nship b etw een unit- t est statemen t co verage and s ystem-test faults later attributed t o those units. Presen t work has b een orga nized in 4 sections the section 2 giv es the notatio n and preliminaries of softw are reliabilit y in terms testabilit y of a mo dule. In the 3 rd sections w e hav e deriv ed the probabilit y of correct output of a particular system corresp onding to a particular input considering b o t h the case presence a nd absence of b enign failure. In Section 4 w e pres ent a brief discussions about the pro cedure men tio ned here. 2 2 Notations and Preliminaries There is no rigo rous definition of ’Q ua lity’. But it can be we a kly define d as the fit- ness of purp o se of any pro duct to its users. Similarly soft w are quality is defined as the confor mance to explicitly stated functions and p erforma nce requiremen ts, explicitly do cumen ted dev elopmen t standards and implicit c haracteristics that are exp ected of all professionally crafted soft w are(Cai Kai-Y uan Cai (3)). Alternatively , t he qualit y of a soft ware ma y b e c haracterized b y some quality factors of a soft w are - reliabilit y , effi- ciency , correctnes s, usability , testabilit y etc. Reliabilit y o f a soft w ar e system ma y b e view ed as the exp ected v alue of pr o babilit y of failure-free op eratio n of a program for a randomly c hosen set of input v ariables. The term failure in t he con text of soft ware reliabilit y implies a r esult other than what was exp ected from the softw are for a set of inputs. F ollo wing V oas et. al. (1995) w e define the t estabilit y of a part icular system as the probability o f failure o f the system fo r a par- ticular input when it is assumed that there is at leas t one fa ult in the syste m. Supp ose w e hav e a soft ware system whic h can b e decomp osed in N mo dules. Thus the testability of a par t icular mo dule, say i th ( ∀ i = 1 (1) N ) mo dule, is giv en b y p i = Prob[ that the i th module will giv e incorrect o utput | there is at least o ne fault, probabilit y distribution of input] (1) The expression for the probabilit y that the i th mo dule will con t a in error if the mo dule has tested n i times successfully , is g iv en b y t he follo wing (Y ang et. al. ( 1 998)) α i ( t ) = α i (0)(1 − p i ) n i α i (0)(1 − p i ) n i + 1 − α i (0) (2) where α i (0) is the probability o f failur e of the system b efore testing. Let π t ( x ) is the probabilit y of a system giving c o rrect output corresponding to a particular set of input x . The expression of π t ( x ) by assuming the indep endent s etup is giv en b y (Y ang et. al. (1998)) π t ( x ) = Y i ∈ S (1 − q i α i ( t )) (3) where q i is the rev ealibility of the i th mo dule and S ( x ) is t he set of those mo dules whic h will be executed b y the input x . The reliabilit y of a soft ware system is giv en b y R t = Z x ∈ X π t ( x ) φ ( x ) dx (4) where X is the set of all p ossible inputs and φ ( x ) is the proba bility distribution of x . 3 Detailed Expressio n of π t ( x ) for Dep endent Setup A softw are system is necessarily an iterativ e. In eac h iteration a particular mo dule accepts a v alue and pro duce an output. The outcomes o f an individual iteration may 3 b e: i) succes s, i.e., the deliv ery of a correct result, ii) a b enign failure of the program, i.e., an output that is not corr ect but do es not, b y itself, cause the en tire mission of the controlled system to fa il, or iii) a cata strophic fa ilure, i.e., an output tha t causes the immediate failure of the entire mission. The c ha racterization of failures in b enign and catastrophic is discussed with example by Bondav alli. et. al. (). In this section w e deriv e t he expression of π t ( x ) first of all only considering the catastrophic failur e and then in the subsequen t subsec t ion considering the b enign and catastrophic f ailure sim ultaneously . 3.1 Expression of π t ( x ) : No Benign F ail ure in the S ystem Consider the ab ov e softw are system with N mo dules. Let p ij b e the probabilit y that the con trol fr om the i th mo dule will b e transferred to the j th mo dule with correct execution ( ∀ i = 1(1) N , ∀ j = 1(1) N ). Let S b e a stat e of successful completion of the system. As S is ac hiev able f rom any one of the mo dule so w e define p iS ( ∀ i = 1 ( 1 ) N ) as the probabilit y of success f ul completion of the mission from the i th mo dule. Here we m ust ha ve p iS + P n j =1 p ij = 1. As w e ha v e a fa ulty system, that is, we hav e a system where there is at least one fault or if the faults can b e classified into categories then there a re at most one fault of each category . So w e intro duce another state F , i.e., unsuccessful completion of the mission. As any mo dule ma y b e faulty so the stat e F a lso can b e ac hieve d from an y of the mo dule. W e define p iF as the probability of unsuccessful completion of t he mo dule i ( ∀ i = 1( 1) N ). The transition probabilit y matrix tak es the following form for the ab ov e setup. Q =           p 11 (1 − α x 1 ( t )) p 12 (1 − α x 1 ( t )) ... p 1 N (1 − α x 1 ( t )) p 1 S (1 − α x 1 ( t )) α x 1 ( t ) p 21 (1 − α x 2 ( t )) p 22 (1 − α x 2 ( t )) ... p 2 N (1 − α x 2 ( t )) p 2 S (1 − α x 2 ( t )) α x 2 ( t ) ... ... ... ... ... ... p N 1 (1 − α x N ( t )) p N 2 (1 − α x N ( t )) ... p N N (1 − α x N ( t )) p N S (1 − α x N ( t )) α x N ( t ) 0 0 ... 0 1 0 0 0 ... 0 0 1           (5) where α x i ( t ) is the proba bilit y o f fault y completion of the i th mo dule for the input x. The expression of α x i ( t ) is giv en b y α x i ( t ) = q i α i ( t ) (6) If we assume tha t the first blo c k is the con trol blo c k then the probabilit y o f correct completion of the mission for the giv en input x is giv en b y (Parze n (1962)) π t ( x ) = N X i =1 ( I N − ˆ Q ) − 1 1 i p iS (1 − α x i ( t )) (7) where ˆ Q is the sub-matrix of Q deleting its last t w o columns a nd rows. 4 3.2 Expression of π t ( x ) : Benign F ailure and Catastrophic F ail - ure are in the System F rom the softw are viewpoint solely , and without referring to any sp ecific application, w e assume here that all detected f ailures (default safe v alues o f the control outputs from the computer) do not prev en t the mission to con tinue and a r e in this sense benign, where a s undetected failures are conserv ativ ely assumed to hav e a ” catastrophic” effect on the con trolled system. Ob viously , if kno wledge of t he consequences of softw are failures on the system was av ailable for a sp ecific system, t he prop er splitting of soft ware f a ilures in to b enign a nd catastro phic could b e precisely ma de. W e mak e the follow ing assumption to model the system. Supp ose S S is a state where the total system, t hat is all the N mo dules, runs without an y fault of either kind. Let B i b e the state where the system is r unning in b enign failure of i th lev el, that is after i iteratio ns the system will en ter in the state S S . As the previous subsection S and F denotes the successful completion of the mission and c o mpletion of the mission with a f ailure resp ectiv ely . The mission will fail if their is a catastrophic failure in the system. Let us also assume that if there is a b enign failure of length gr eater than a threshold v alue, sa y n c , then the system will enter in a catastro phic failure region. Although this assum pt io n will tak e the mo del a little aw a y from realit y , a mo del should b e go o d enough to handle a b enign failure of any arbitrary random length, but this assumption will make the calculation of reliabilit y expression easier whic h will increase its practical application. A t this p oint note tha t the state S , that is the success f ul completion of the prog r am, can b e ac hiev ed o nly from the state S S , where as the state F can b e ac hiev ed from any of the state S S or B i ’s ( ∀ i = 1 (1) N ), but w e assume here the con trol will b e transferred from the state B i to B i − 1 only to reduce the n um b er of parameters in the mo del. The transition probabilit y matrix will be as follows Q =                Q 00 Q b 01 Q b 02 ... Q b 0( n c − 2) Q b 0( n c − 1) Q b 0 n c S 0 F 0 Q b 10 O O ... O O O ¯ 0 ¯ 0 O Q b 21 O ... O O O ¯ 0 ¯ 0 ... ... ... ... ... ... ... ... ... O O O ... Q b ( n c − 1)( n c − 2) O O ¯ 0 ¯ 0 O O O ... O Q b n c ( n c − 1) O ¯ 0 ¯ 0 ¯ 0 ′ ¯ 0 ′ ¯ 0 ′ ... ¯ 0 ′ ¯ 0 ′ ¯ 0 ′ 1 0 ¯ 0 ′ ¯ 0 ′ ¯ 0 ′ ... ¯ 0 ′ ¯ 0 ′ ¯ 0 ′ 0 1                (8) Here the matrix Q 00 is a N × N matrix whic h describ es that the flow is running without en tering in b enign failure or cata strophic fa ilure. The matrix Q b 0 k is also a N × N matrix giving the transition probabilit ies of the flo w of control from stable state to the k th lev el b enign failure ( ∀ k = 1(1) n c ). Similarly , the matrix Q b k l whic h is also N × N denotes the transition probabilities of the con trol ente ring from t he k t h lev el b enign f a ilure to l th lev el ( ∀ k = 1(1) n c ∀ l = 1(1) n c ). F rom the k th lev el b enign failure w e can only ac hiev e the k − 1th leve l b enign failure so Q b k l = O ( ∀ l 6 = k − 1 ). Where O is t he n ull matrix of order N × N . S 0 is a N × 1 v ector of the transition probabilities of successful completion of the mission from the stable state. As the mission can terminate success f ully only from 5 the stable stat e so t he r est of the en tries in this column are a ll zero. ¯ 0 denotes a n ull v ector of length N and ¯ 0 ′ denotes transp ose of ¯ 0. Finally , F 0 is a column v ector of length N g iving probabilities of reac hing the state of catastrophic failure f r o m the stable stat e. T o give the structure of sub-matr ices Q 00 , let us define p S S ij b e the probabilit y of the con trol to en ter from the i th mo dule to j th mo dule in the state S S . So the matrix Q 00 is giv en by Q 00 =      p S S 11 p S S 12 ... p S S 1 N p S S 21 p S S 22 ... p S S 2 N ... ... ... ... p S S N 1 p S S N 2 ... p S S N N      (9) Let us also define p S B ij b e the probabilit y tha t the control will b e transferred from the mo dule i to the mo dule j from the state S S to an y of b enign f ailure. Let also p B k the probabilit y that the con tro l will en ter in B k , th us the probability that the con trol will en ter in the j th mo dule from the i th mo dule in the state B k is giv en b y p S B ij p B k . So the matrix Q b 0 k will take the follo wing form Q b 0 k =      p S B 11 p B k p S B 12 p B k ... p S B 1 N p B k p S B 21 p B k p S B 22 p B k ... p S B 2 N p B k ... ... ... ... p S B N 1 p B k p S B N 2 p B k ... p S B N N p B k      (10) If p iS and p iF is resp ective ly the successful completion of the mission and ac hieving catastrophic failure from the i th mo dule. Then w e m ust ha ve N X j =1 p S S ij + n c X k =1 p B k N X j =1 p S B ij + p iS + p iF = 1 ∀ i = 1(1) N (11) The ma t r ix Q b k k − 1 tak es the following form Q b k k − 1 =      p bb 11 p bb 12 ... p bb 1 N p bb 21 p bb 22 ... p bb 2 N ... ... ... ... p bb N 1 p bb N 2 ... p bb N N      (12) Here w e ha v e N X j =1 p bb ij = 1 ∀ i = 1(1) N (13) Finally , the matrix Q b 10 is the matrix of tra nsition proba bilit ies, s ay p bS ij , that the flo w of con trol will b e transferred f r o m t he i th to the j th module and from the B 1 to S S . Here also N X j =1 p bS ij = 1 ∀ i = 1(1) N (14) 6 By ass uming as b efo re the first module as the con tr o l module the expression of π t ( x ) is giv en π t ( x ) = N X i =1 ( I N n c − ˆ Q ) − 1 1 i p iS (15) where ˆ Q is once again the sub-mat r ix of Q deleting its last t w o columns and ro ws. 4 Conclus ions In this work w e hav e g iv en an expression of the reliabilit y of a softw are system whic h can b e divided in a finite num ber of mo dules. The transition pr o babilities we hav e considered can b e easily estimated using maxim um lik eliho o d metho d of estimation. Consider the setup without b enign f ailure, supp ose i th blo c k is tested n i times, out of whic h x i j times t he control is transferred to the j th state ( ∀ i = 1(1) N & ∀ j = 1(1) N , S, F ). The maxim um likelihoo d estimates of p ij is x i j / ( P N i =1 x i j + x i S ) and that of α x i ( t ) is x i F /n i . Hence estimate o f π t ( x ) can b e o btained and let it b e denoted b y ˆ π t ( x ). Finally the estimate of reliability of a system c an b e g iv en b y ˆ R t = 1 | W | X x ∈ W ˆ π t ( x ) (16) where W is the set of all inputs whic h are used for testing. This is an extension of some previous w ork and t he mo del what we hav e considered are mo r e r ealistic fo r some control soft ware whic h are designed to detect its own internal errors and then issue a safe output and r eset itself to a kno wn state from whic h the program is lik ely to pro ceed correctly . References [1] A.Bertolino and L.Strigini (1996). On The Use of T estability Measures for Dep end- abilit y Asse ssmen t. IEEE T ra ns on Sof tw are Engineering 22(2):97- 108. [2] C. Beightle r and D.T. Phillips (1976). Applied Geometric Prog ramming. Jo hn Wiley & Sons,Inc., New Y ork. [3] E.Cinlar (1975). In tro duction to Sto c hastic Processes. Pren tice-Hall, Inc., Engle- w o o d Clios, N.J. [4] E.C.Soistman and K.B.Rag sdale (1984). Com bined Hardw are/Sof t ware Reliability Prediction Metho dolog y . R ome Air Developme nt Cente r Con tract Report OR. 18 - 173, V o l.2. [5] E.P arzen. (1962), Sto c hastic Pro cesses , Holden-Da y , San F ra ncisco, Calif. [6] J.H.P o ore, H.D .Mills and D.Mutchle r (19 93). Planning and Certifying So ft ware Sys- tem Reliability . IEEE Soft w a re, 88 -99. 7 [7] J.M.V oas and K.W.Miller ( 1995). Soft ware T estability : The new v erification. IEE E Soft ware. 17-28. [8] J.Ra jgopal and D.L.Brick er (1995). An Algorithm f or Solving The P olynomial GP Problem, Based on G eneralized Prog ramming. Departmen t of Industrial Engineer- ing, Univers it y of P ittsburgh, T ec hnical R ep ort No.TR95-10. [9] J.Ra jgopal and M.Mazumdar (19 9 5). D esigning Comp onen t T est Plans for Series System Reliability via Mathematical Programming. T ec hnometrics 37,195-21 2. [10] J.Ra jgopal and M.Mazumdar (1997). Minim um Cost Comp onen t T est Plans for ev aluating Reliability of a Highly Reliable P a r a llel System. Nav al Researc h Logistics 44, 401 -418. [11] K.S.Al-Sultan, M.F.Hussain and J.S.Nizami (19 9 6). A G enetic Algorithm for The Set Co v ering Problem. The Journal of t he O p erational Researc h So c. 47, 5 , 702-70 9. [12] K.Siegrist(1998). Reliability of Systems with Marko v T ransfer of Control. IEEE T ransaction o n Sof tw are Engineering 14, 10 49-1053 . [13] K.W.Miller, L.J.Morrel, R.E.Noona n, S.K.P ark, D.M.Nicol, B.M.Murril and J.M.V oas (1 9 92) Estimating the probabilit y of failure when testing rev eals no f ailure. IEEE T rans. on Softw are Engineering 18(1): 33-43. [14] Mark C.K. Y a ng, W.Eric W ong, Alb erto P asquini (1998). Applying T estabilit y to Reliabilit y Estimation, Proc. of IEEE In ternational Symposium on Sof t ware Relia- bilit y Engineering, P uderb orn 90-99. [15] M.Avriel,R.S.Dem b o and U.Plassy (1975). So lut io n of Generalized Geometric Pro- grams. International Journal for Numerical me tho ds in Engineering 9, 149-1 68. [16] R.C.Cheung ( 1 980). A Use r -Orien ted Reliabilit y Mo del. IEEE T rans. Softw are En- gineering, SE- 6(2): 118-1 2 5. [17] R.C.Easterling, M.Mazumdar, F .W.Sp encer a nd K.V.Diegert (1991). System Based Comp onen t T est Plan a nd Op erating Characteristics: Binomial Data T ec hnometrics 33,287- 2 9 8. [18] S.Gal (1974). Optimal T est Design for Reliabilit y D emonstration. Op erational Re- searc h 22, 1236- 1242. [19] S.Ghosh, A.P .Math ur, J.R.Ho rgan, J.J.Li, W.E.W ong(1997). Softw are F ault Injec- tion T esting on a D istributed System - A Case Study , Pro c. of the 1st Inte r national Qualit y W eek Europ e, Brussels , Belgium. [20] S.W olfram(1996) . The Mathematics Bo ok (3r d edition). Cam bridge Univ ersit y Press and W olfram Media, Inc.Champaign, 3. [21] W.Kuo (1992). Softw are Reliabilit y . Mayn ards Industrial Engineering Handb o ok, 4th edition(W.K.Ho dson, Editor-in-Chief ), 11116-111 22. 8 [22] W.Eric W ong, J.R.Horg an, S.London and Adity a P .Mathur (1998). Effect of test set minimization on F ault Detection Effectiv eness. Soft ware-Practice and Exp erience, 28(4): 347- 369. 9