A Combinatorial-Probabilistic Diagnostic Entropy and Information

1 A Combinatorial-Probabilistic Diagnostic Entropy and Information Henryk Borowczyk, Member, IEEE Air Force Institute of Technology, Warsaw, Poland borowczyk@post.pl Abstract A new combinatorial-probabilistic diagnostic entropy has been introduced. It describes the pair-wise sum of probabilities of s y stem conditions that have to be distinguished during the diagnosing process. Th e proposed measure describes the unc ertainty of the s ystem conditions, and at the same time complexit y of the diagnos is problem. Trea ting the assumed combinatorial- diagnostic entrop y as a primar y notion, the information delivered by th e symptoms has been defined. The r elationships have b een derived to facilitate explicit, quantitative assessment of the information of a sin gle s y mptom as well as that of a s y mptoms set. I t has been proved that the combinatorial-probabilistic information shows the property of additivity. The presented measures are fo cused on diagnosis problem, but they can be easil y applied to other disciplines such as decision theory and classification. Index Terms -- entropy, fault diagnosis, information, multi -valued model, uncerta int y I. INTRODUCTION Constructing an optimal set of diagnostic s y mptoms/tests and optimal sequence of gathering/execution thereof is one of the most important problems in engineering systems diagnosis [ 1 ]  [7]. Applied optimization method depends on the form of diagnostic model and optimization criterion [1],[2],[5],[8 ]  [12]. The diagnostic model de scribes the relationship between a s ystem condition (the set of faults and healthy condition) and diagnostic symptoms [4],[5],[13 ]  [16]. Most models use binary (g ood  no-good) conditi ons and binary (nor mal  abnormal) symptoms [3],[8],[17]. Better results can be obtained using the qualitative, approximate, a nd multi-valued models [ 2],[4],[15],[18],[19]. In [2], it is proved that the length of diagnosis algorithm in the case of multi-valued system conditions and multi-valued symptoms is not larger than the binary algorithm. Qualitative and multi-valued models can be applied to determine a diagnosis al gorithm [1],[2],[19], approximate inference within expert sy stems [ 4],[15],[19 ]. One of the methods of constructing a diagnosis alg orithm consists in a pplying the information - based analysis [2],[ 3],[8],[16],[18],[20 ]  [24] , that is, description of the s y stem condition uncertainty and the amount of information delivered by means of individual symptoms and sets thereof. This aim can be reached with the Shannon-introduced quantities: the entrop y, and the amount of 2 information [25]. There are some other kinds of entropies [27 ] that can be considered  Renyi  s entropy [26],[ 28]    - entropy [29] , a nd functions () zt  [30]. Characterization of information measures (from the information theory point of view) have been extensivel y discussed in [31],[32]. The abovementioned ent ropies were introduced f or solving inform ation-theoretic problems (e.g. coding). For other problems, different forms of entrop y mi ght be more suitable [26],[28] ,[33]. Therefore, finding the form of entropy best tailored to meet the diagnostic require ments is justified. This article deals with a multi -valued diagnostic model that exploits the multi-valued system conditions and multi-valued symptoms, where the set of values taken b y conditions and symptoms is finite. The set of desirable properties of the proposed diagnostic entrop y is determined b y taking the diagnostics point of vi ew into account. The problem is formulated in a s y stem condition-set partition framework [33 ]. The organization of the article is as follows. In section II, basic assumptions concerning multi- valued diagnostic model are stated. Section III describes information-theoretic, set-partition framework for the diagnosis algorithm designing. I t is a start ing point for establishing a set of postulated properties of diagnostic entropy that is described in Section IV. In Section V, the combinatorial-probabilistic diagnostic entropy is introduced and its postulated properties are proved. The combinatorial-probabilistic diagnostic information of symptoms and sets thereof is defined in Section VI. II. ASSUMPTIONS Further considera tion is conducted with the following assumptions formu lated and referring to a multi-valued model of the system under the diagnosing. 1. A finite set of the system conditions is determined: { }, 1 , ... , i E e i n  (1) Elements i eE                s y stem under diagnosis is in the i-th condition      E - as a certain event. Conditions might be one -fault or multi-fault ones. 2. The s y stem can remain in one, and only one, of the condition i eE  with the probability () i Pe , and the following holds: 1 , , ( ) 0 , ( ) 1 i in P e P E    K (2) Probabilities () i Pe estimations can be provided b y real -world experiment or by using the well - known engineering procedure of the Fa ilure Modes and Effects Anal y sis [ 34]. 3. Determined is a finite set of symptoms: { } , 1 , .. . , r D d r t  (3) 3 and a finite set of  values { 0 , , 1 } A   K (4) Multi-valued operator (. ) R mapping ED  into A is crisp, fuzz y , or r ough one [2],[18],[35],[36]. If the s y stem is in the condition i e , a value of the symptom r d is denoted in the following manner: ( / ) , r i ir ir R d e g g A  (5) 4. For all the symptoms, the following holds: [ ( / ) ] 1 r i ir r i ir d D e E E P R d e          (6) It means that dia gnosis is noiseless [ 37 ] or diagnostic model consists of all real-wo rld instances representing class of condition i eE  . 5. The multi-valued model of the s ystem has b een presented in the form of a diagnostic matrix G :   ir n x t Gg  (7) where: ( / ) , ir r i ir g R d e g A  (8) The assumed form of the model and the assumptions 1  5 offer good representation of many actual systems under the diagnosing [2 ], [5]  [7 ], [17 ], [38 ], [39]. III. AN INFORMATION-THEORETIC, SET-PARTITION FRAMEWORK FOR DESIGNING A DIAGNOSIS ALGORITHM If the s ystem can remain in any of the conditions i eE  with the probability () i Pe , the Shannon entropy () HE thereof is expressed with the following relationship: 1 ( ) ( ) l o g ( ) n ii i H E P e P e     (9) Value 2   is usually assume d as the base of a logarithm. The general form (9) is more suitable and convenient, if the multi-valued ( 2)   assessment of values of symptoms is a requirement. The relationship (9) describes the initial uncertainty of the system condition, that is, prior to the selection of an y s ymptom. Assume that any r dD  of the symptoms has been selec ted as the first one. The set of values thereof takes the fo rm (4). Ther efore, subsets () jr E d E  can be distinguished. 0, , 1 ( ) { : ( / ) ; 1 , , , } jj j r i r i j j j E d e R d e j i n j A        L L (10) To sim plify the notation, indices have been assigned to the subsets () jr E d E  , the indices being consistent with suitable values of the symptom r dD  . 4 On the assumption (6) t hat there is full li kelihood of the value of the symptom r dD  , the following are satisfied: 1 , 0,..., 1 0 11 0 0 1 a) ( ) ( ) , ( ) b) , 1 , ( ) j j j j r l r j r jl j jl n j j j i j j i E d E d E d E n n p p P e                       U (11) What follows from (11) is that the s elected symptom r d induce s c onditions set partition, which is denoted in the following manner: 01 { ( ) } { ( ) , , ( ) } j r r r E d E d E d    L (12) If the s ymptom r d takes the value jA  , then according to (10) and (11 ), the system can find itself in one of the condition s () j i j r e E d  with the probability: () () j j i i j Pe Qe p  (13) Provided that the actual condition * e of the system has been identified with accurac y to the subset () jr Ed , the system entropy equals: 1 ( / ( )) ( ) log ( ) j jj j n j r i i i H E E d Q e Q e     (14) Because the s ymptom r d can take any v alue jA  with the probabilit y j p , the ave rage entrop y after having selected the symptom r d equals: 1 0 ( / ) [ / ( ) ] r j j r j H E d p H E E d      (15) where: ( / ) r H E d  the average entropy after having selected the sy mptom r d . The amount of information on the system condition contained in the symptom r d is described as the difference between the entropies () HE and ( / ) r H E d : ( ) ( ) ( / ) rr J d H E H E d  (16) As the conditional entropy ( / ) r H E d is not larger than the initial entropy () HE , the information () r Jd is a non-negative quantity : ( ) 0 r Jd  (17) After having selected the first s y mptom, the conditional entropy ( / ) r H E d is usually larger than zero. Therefore, sele cting further symptoms proves indispensable. The amount of information entered with the s y mptom selected in the k- th turn is found from the following general relationship: ( ) 1 1 ( ) ( ) ( / ) ( / , ) k k k k J d H E D H E D d   (18) where: () k d  symptom selected in the k- th turn, 5 1 k D   a set 1 k  of symptoms selected prior to symptom () k d . The information-based method consists in subsequent selections of symptoms that contain maximum information on the system condition: 1 ( ) 1 1 \ ( / ) m ax ( / ) rk k k r k d D D J d D J d D     (19) The process is stopped when the remaining entropy is e qual to z ero or the amount of information of all remaining symptoms equals zero. One should e mphasi ze that the terms diag nostic entrop y and diagnostic information are not the same. Dia gnostic entropy is a feature characterizing uncertainty of the s ystem condition and/or complexity of diagnosis algorithm c onstructing process . Diagnostic information c haracterises quality of diagnostic symptoms/tests. The above-presented information-based method of finding a set of s y mptoms proves to show at least two features of significance:  the criterion of selecting the s y mptoms is defined on the basis of a primary term, that is, the entropy, which renders the possibilit y of gaining coherent measures of the uncertaint y of the condition and the amount of information entered by the s ymptom;  it uses the property of information additivity  the total amount of information on the s y stem condition, gained after the set of K s ymptoms has been selected, equals the sum of conditional information of individual symptoms: ( ) 1 1 ( ) ( / ) K K k k k J D J d D     (20) However, it appears that the application of the criterion function of selecting the s y mpt oms in the form of the amount of information does not lea d to optimal solutions, even in some specific cases, when the set of available symptoms is unlimited. On the other hand, the idea of information-b ased approach to the question of optimi zing the set of symptoms seems worth y of notice b ecause of the above -mentioned features. A question a rises: Is it possible to apply a similar approac h on the g rounds of some function different than the Shannon entropy, a function that describes the uncertaint y of the s ystem condition; and if so, what properties should it then have? Additivity is the basic fe ature of the Shannon entropy, which determines the logarithmic form thereof. For two statisti cally ind ependent s ystems this feature can be written in the following form: 12 ( ) ( ) ( ) H E H E H E  (21) where: () HE  entropy of a complex s y stem O ; () i HE  entropy of the system i O . The two systems are statistically independent if acquisition of information on the condition of one of them does not affect the probability distribution of conditions of the other system. In diagnostics, the fact of the 1 O and 2 O systems remaining independent can be interpreted as the symptoms observed with one of the systems remaining independe nt of the condition of the other system. Such being the case, the question of condition identification of a complex system resolves itself into two independent questions of condition identifica tion of component systems. Finding a solution to the issue formulated in this way does not require the me asure of the uncertainty of the complex sy stem condition to be de fined; therefore, there is no need to satisfy the condition of 6 additivity. What results is a conclusion of great significa nce to further considerations: the function that describes the uncertainty of the c ondition of the system under dia gnosis does not need the additivity feature in the sense described with (21). This permits to quit the function that shows the logarithmic fo rm, that is, the Shannon entrop y. In thi s wa y , the first o f the above -formulated questions has been affirmatively answered. The question of properties reads as follows: What properties should a function that describes the uncertainty of the condition show to make it a basis for the info rmation-like approach for optimizing a set of symptoms? And what form should this function take? The func tion searched for will be further on c alled the combinatorial-probabilistic entropy and denoted with () B HE . IV. POSTU LATE D PROPERT I ES OF THE COMBINATORIAL-PRO BABILISTI C DIAGNOSTIC ENTROPY First and foremost, it should be noted that the combinatorial-probabilistic entropy () B HE will find its application when designing the diagnosis alg orithm. Therefore, this measure should take into acc ount specific features of that process. In pa rticular, it should facilitate quantitative assessment of the partition of the conditions set induced by the selected symptoms. A set of postulated p roperties of () B HE        Shannon entropy. As it follows from (9), probabilities of the s ystem condition s () i Pe are arguments of the function of entropy. In some indirect way, this function also depends on the cardinalit y of the conditions set () n card E  . This dependence becomes more evident if the probabilities of all the condition s are ( ) 1 / , 0 , , i P e n i n  K . Such being the case, the function of entropy takes the following form: ( ) lo g H E n   (22) However, what most significantly affects the value of entrop y is the probabilities of the conditions. This can be illustrated with the exemplar y two systems: the sets of conditions thereof as well as probabilities of the conditions are defined in the following way: 1 2 1 2 ' ' ' ' ' 1 2 3 4 5 ' ' ' 1 2 3 '' 45 { , } ( ) ( ) 0, 5 ' { , , , , } ( ) ( ) 0.05 , ( ) 0, 84, ( ) ( ) 0 , 03 E e e P e P e E e e e e e P e P e P e P e P e         Entropies of these s ystems, at 2   , a re ( ) 1 , 0 HE  and ( ' ) 0 , 94 7 HE  , re spectivel y . It means that the entropy of a two-condition system is higher than that of a five-condition system. On the other hand, well known is the fact that one sy mp tom is enoug h to identif y th e condition of the first system, whereas identi fication of the second system condition needs at least three symptoms. Therefore, it seems reasonable to postulate that the demanded combinatori al-probabilistic entropy () B HE be at the same time the function of the cardinality of the conditions set () n card E  and probabilities of condition s () i Pe : 7 ( ) ( , ) B H E f n P  (23) where: { ( ) } , 1 , , i P P e i n  K What else should be expec ted is the monotonic increase in the value of function (23) with the growth of n : ( , ) ( ' , ' ) ' f n P f n P n n    (24) Two other properties result from conditions of setting () B HE to zero. If it is known a priori that the set of s y stem c onditions is one-component only ( 1 n  ), the condition of the system is then definitely determined and () B HE should take value zero: 1 ( ) 0 B n HE   (25) Another extreme ca se takes place when the selected s y mptoms induce pa rtition of the conditions set in the form of one-component subsets { { } } , 1 , i e i n  K . It means that all the pairs of conditions have been distinguished by the sel ected sy mptoms; therefore, the uncertainty o f the condition equals zero: ( / {{ } } ) 0 Bi H E e  (26) The demanded function is imposed on with the condition of linearity against the assembly o f variables P in the following form: 12 ( , ( ) , ( ) , ... , ( )) ( , ) n f n c P e c P e c P e c f n c P  (27) where: c is the nonzero constant. Because the p roperty ( 27 ) has no equivalent among properties of the Shannon entrop y , more detailed consideration of resulting consequences seems indispensable. Ass ume that given is some partition of the conditions set in the form (12). I f an actual condition * e of the sy stem is identified with accuracy to the subset j E , the uncertainty of the condition c an be written in the following manner:   ( / ) , B j j j H E E f n Q  (28) where: 12 ( ) ( ) ( ) , , ... , j j j n j j j j P e P e P e Q p p p       With (27) taken into account, the relationship in (28) takes the following form: 1 ( / ) ( , ) j B j j j H E E f n P p  (29) or ( / ) ( , ) j j B j j p H E E f n P  (30) where:   12 ( ), ( ), ... , ( ) j j j j n P P e P e P e  8 The average uncertaint y of the system condition, at the partition in (12) assumed, results from the definition of the statistical mean value: 1 1 ( / { }) ( / ) B j j B j j H E E p H E E      (31) With the following notation introduced: ( ) ( , ) j B j j H E f n P  (32) and (30) taken into account , the relationship in (31) can be finally written in the following form: 1 1 ( / { }) ( ) B j B j j H E E H E      (33) The relationship (32)     j - th subset j E to the average uncertaint y of the s ystem condition ( / { }) Bj H E E . Consequently , ( .) B H at each stage of the optimiz ation process can b e found by means of the p robabilities a p riori () i Pe , with no requirement to find conditional probabilities () i Qe . I n other words, imposing condition (27) on the ( .) B H , means some reduction in labor demand while carrying out computations to determi ne the optimal set of symptoms. V. THE COMBINATORIAL-PROBABILISTIC DIAGNOSTIC ENTROPY It follows from the above-presented considerations that the combinatorial-probabilistic entropy () B HE of the system under diagnos is should have the following properties: ( ) ( , ) B H E f n P  (34) ( ) ( ' ) ' , ( ) , ' ( ' ) BB H E H E n n n C a rd E n C a r d E      (35) 1 ( ) 0 B n HE   (36) 1 ( / { }) ( ), ({ }) m B j B j j j H E E H E Ca rd E m    (37) ( / {{ } }) 0 , 1 , . .. , Bi H E e i n  (38) The form of the function ( , ) f n P ca n be defined with two methods. The first one consists in the formal deduction of the function on the bas is of t he set of post ulated properties. This approach is followed when there is s ignificant proof that there exists only one fun ction showing the p reset properties. Th e second method, usually much si mpler, consists in the a rbitrary acceptanc e of a certain form of the function and in proving that it shows the postulated properties. As there are no limitations put in this article on the number o f fun ctions that sa tisf y conditions (34)  (38) the latter of the methods is applied.                combinatorial- probabilistic entropy ( .) B H and the process of planning the diagnostic algorithm should be a 9 reasonable prerequisite for the selection of the form of ( .) B H . Further consideration is based on the following theorem: Theorem 1 If given are: a) a finite set of system condition s { }, 1 , ... , i E e i n  (39) b) probabilities of condition s 1 , , ( ) 0 i in Pe    K (40) with 1 ( ) ( ) 1 n i i P E P e    (41) then function 1 11 ( ) ( ( ) ( )) nn B i j i j i H E P e P e       (42) which determine s the sum of proba bilities of all unordered pairs of system conditions shows the postulated properties (34)  (38). Proof (draft) To prove Theorem 1 lemmas 1  6 are used. Lemma 1 If assumptions (39)  (41) have been satisfied, the following relationship proves true: 1 1 1 1 ( ( ) ( )) ( )( 1 ) n n n i j i i j i i P e P e P e n            (43) Proof (see APPENDIX) On the grounds of the Lemma 1, the relationship in (42) can be presented in the equivalent form: 1 ( ) ( ) ( 1 ) n Bi i H E P e n    (44) Lemma 2 Equivalence (35) holds for the function (42). Proof (see APPENDIX) Lemma 3 For a one-condition system, function (42) takes value equal to zero. Proof (see APPENDIX) Lemma 4 10 If given are: a) a proper subset j EE  , such that: { } , 1 , .. ., j j i j j E e i n  (45) b) probabilities a priori of the condition s: 1 ( ) 0, ( ) j jj ij j j n i i j eE i P e P e p      (46) then the following equality holds: 1 11 ( ( ) ( )) ( 1 ) jj jj j j j nn i k j j i k i P e P e p n         (47) Proof (see APPENDIX) On the grounds of the Lemma 4, the following notation can be introduced: 1 11 ( ) ( ( ) ( )) ( 1 ) jj jj j j j nn B j k k j j i k i H E P e P e p n          (48) Lemma 5 If given is some partition of the conditions set { }, 1 , . . ., j E j m  such that: 1 , , 1 () j j j n ij jm i P e p      K (49) 11 ( ) 1 j j j n m i ji Pe    (50) 1 m j j nn    (51) then the following equality holds: 1 ( / { }) ( ) m B j B j j H E E H E    (52) Proof (see APPENDIX) Lemma 6 If the pa rtition of the conditions set is given in the form of one-component subsets { { }} , 1 , .. ., i e i n  , then the uncertainty of the system condition equals zero. ( / { { }} ) 0 Bi H E e  (53) Proof (see APPENDIX) What has been shown by means of the lemmas 1  6 is that function ( 42 ) shows the post ulated properties (34)  (38), which completes the process of proving Theorem 1. It comes from the above-presented consideration that function (42), wh ich defines the sum o f 11 probabilities of all unordered pairs of conditions of the s ystem under t he diagnosing, can be accepted as the combinatorial-probabilistic entropy . VI. THE COMBINATORIAL-PROBABILISTIC D IAGNOSTIC INFORMATION The initial uncertainty of the system condition (prior to the selection of any sy mptom) is equal to: ( ) 1 B H E n  (54) If an y s y mptom r dD  , with its set of values as in (4) has been selected as the first one in the sequence, it induces the partition: 01 { ( ) } { ( ) , , ( )} j r r r E d E d E d    L (55) With full likelihood of the value of symptom r d satisfied then: 1 , 0 ,..., 1 0 11 0 0 1 a) ( ) ( ) , ( ) b) , 1 , ( ) j j j j r l r j r jl j jl n j j j i j j i E d E d E d E n n p p P e                       U (56) In compliance with the Lemma 4, the uncertaint y of the s y stem condition, after selection of symptom r d that induce s partition (55), equals : 1 ( / { }) ( 1 ) m B j j j j H E E p n    As the partition (55) is explicitly defined b y means of the symptom r d that induce s it , the above equation can be written as : 1 0 ( / ) ( 1 ) B r j j j H E d p n      (57) It is easy to observe th at the uncertaint y of the s ystem condition after the selection of the symptom r d is not greater than the initial uncertainty , that is: ( ) ( / ) B B r H E H E d  (58) Equality in (58) occurs in the case described with the following condition ( ) ( / ) ( 1 ) B B r j j jA H E H E d n n p        (59) It means that the value of the s y mptom r d does not depend on the system co ndition, and such a symptom remains useless as far as the condition identification is concerned. On the grounds of equations (58) and (59), the noti on of the combi natorial-probabilistic information of a symptom can be defined. Definition 1 The combinatorial-probabilistic information of the s ymptom r d is equal to t he difference in the uncertainty of the condit ion before this sy mptom has been selected and the unce rtainty l eft after the selection. 12 If the s ymptom r d is selected as the first one in the sequence, then, according to the Definition 1, the following can be written: ( ) ( ) ( / ) B r B B r J d H E H E d  (60) where: () Br Jd is the combinatorial-probabilistic information of the symptom r dD  . After substit uting (54 ) and (57) into (60) and conditions of (56) t aken into account, the combinatorial-probabilistic information can be presented in two equivalent forms: 1 0 1 0 ( ) ( 1 ) ( ) ( ) B r j j j B r j j j J d n p J d p n n           (61) From (58) and (60) it becomes evident that the information () Br Jd can take non- negative v alues only. ( ) 0 Br Jd  (62) The equation (60) and earlier considerations o n the form of the combinatorial-probabilistic entropy () B HE give grounds to formulate the following conclusion: Corollary 1 The combinatorial-probabilistic information () Br Jd is equal to the sum of the probabilities of all unordered pairs of conditions distinguishable because of the symptom r d : 21 0 1 1 1 ( ) [ ( ) ( )] j k jk jk n n B r i i j k j i i J d P e P e             (63) This confirms the coherence of the introduced measures of the sy stem condition uncertainty and the information of symptoms. Selection of one s ymptom usually does not permit condition identification with the required accuracy; therefore, th ere is ne ed for selecting f urther s y mptoms. I f the s dD  s y mptom, with the set of values of the fo rm (4), has been selected as the second one in the sequence, then in each of the subsets () jr Ed of the partition (55) induces the following partition: 01 0 ,.. ., 1 { ( , ), ..., ( , )} j r s j r s j E d d E d d      (64) where: ( , ) { : ( / ) ( / ) , 1 , ..., , } jl jl jl jl r s i r i s i jl jl E d d e R d e j R d e l in      With full likelihood of the value of the symptom s d assumed, the following are satisfied: 13 0 ,.. ., 1 , 0,..., 1 1 0 ,.. ., 1 0 11 0 ,.. ., 1 0 0 1 ) ( , ) ( , ) ) ( , ) ( ) ) , , ( ) jl jl jl jl r s jk r s j l k lk jl r s j r j l n jl j jl j jl i j l l i a E d d E d d b E d d E d c n n p p p P e                                U (65) With the Lemma 4 as the basis and conditions of (56) and (65) taken into account, the relationship that defines the uncertaint y of the system condition after having selected both the symptoms, that is, , rs d d D  , can be written in the following manner: 11 00 ( / , ) ( 1 ) B r s jl jl jl H E d d p n      (66) On the other hand, the form of the combinatorial-probabilistic infor mation of the symptom s d under the condition that the s y mptom r d has been selected as the first one in the sequence, results from the general Definition 1 ( / ) ( / ) ( / , ) B s r B r B r s J d d H E d H E d d  (67) After substituting (57) and (66) into (67), the following is arrived at: 1 1 1 0 0 0 1 1 1 0 0 0 ( / ) ( 1 ) ( 1 ) B s r j j jl jl j j l j j jl jl j jl j l l J d d p n p n p n p n p p                                  The conditional combinatorial -probabilistic information of the symptom s d can be w ritten in two equivalent forms: 11 00 11 00 ( / ) ( ) ( / ) ( ) B s r jl j jl jl B s r jl j jl jl J d d n p p J d d p n n           (68) As there is inequality taking place in a way similar to that of (58) ( / ) ( / , ) B r B r s H E d H E d d  (69) therefore, ( / ) 0 sr J d d   The above considerations can be genera liz ed to the question of defining the combinatorial- probabilistic information of the symptom s dD  under the condition that the earlier selected set of k symptoms k DD  , then ( 1 ) ( 1) ( ) { , , . .. , } kk D d d d  (70) Assuming that a fter all the s ymptoms of the set k D have b een selected, t he partition of the conditions set takes the form 0 1 1 { ( ) } { ( ) , ( ),... , ( ) } k j k k k m k E D E D E D E D   (71) 14 where: k m is the power of the family of subsets, which is the partition of the conditions set E. The following relationships also need be satisfied: )) k kk a m n b m   (72) Equality in (72)a is reached when the symptoms from the set k D induce the (71) partition in the form of one-member subsets, whereas equalit y in the (72)b, when, after selection of each of the symptoms, the cardinality of the famil y of subse ts, increases  -times. The uncertaint y of the system condition after having selected all the symptoms from the set k D equals: 1 0 ( / ) ( 1 ) k m B k j j j H E D p n     (73) If the s ymptom sk dD  is selected as the n ext one successively, then it induce s p artition in the form of (74) in each of subsets of the partition ( 71 ) : 01 0,... 1 { ( , ), ..., ( , )} k j k s j k s jm E D d E D d     (74) where: ( , ) { : ( / , 1 , .. ., } jl jl jl k s i s i jl jl E D d e E R d e l i n     With full likelihood of the value of the symptom s d assumed, the c onditions similar to those of (65) are satisfied, and: 11 0,..., 1 00 , k jl j jl j jm ll n n p p         (75) The uncertainty of the system condition after having selected all the symptoms from the set k D and the symptom sk dD  is equal to: 1 1 00 ( / , ) ( 1 ) k m B k s jl jl jl H E D d p n       (76) Using the g eneral D efinition 1, the c ombinatorial-probabilistic information of the symptom s d , under the condition that earlier the set of symptoms k D has been selected, can be written in the following form: ( / ) ( / ) ( / , ) B s k B k B k s J d D H E D H E D d  (77) After substituting (73) and (76) into (77) and taking (75) into account, the following is arrived at: 1 1 00 1 1 00 ( / ) ( ) ( / ) ( ) k k m B s k jl j jl jl m B k jl j jl jl J d D n p p J d D p n n              (78) It can be easily noticed that relationships (78) are the generalization of (68); they become identical when 1 k  , and k m   . Another issue of significance is to define the information of the set of k symptoms k DD  . In order to do this, the earlier introduced Definition 1 should be gene raliz ed to the following form: 15 Definition 2 The information of the set of s y mptoms () Bk JD is equal to the difference between the initial uncertainty of th e cond ition () B HE and the uncertai nty left after having selected all the symptoms from the set k D - ( / ) Bk H E D : ( ) ( ) ( / ) B k B B k J D H E H E D  (79) After substituting (54) and (73) into (79), the following is arrived at: 1 0 ( ) ( 1 ) k m B k j j j J D n p     (80) What comes out f rom the comparison betw een (61) and ( 80) is that both the relationships take identical form in the case k D is a one-member set. Assuming, that after having selected all the symptoms from the set k D , the condition uncertainty is non-zero and the symptom sk dD  has been selected as the nex t one in the sequence. The joint partition of the se t of co ndition s induced with the symptoms fr om the se t k D a nd sk dD  ha s taken the form (74), whereas the condition uncertaint y is described with the relationship (76). Using Definition 2, the total information of the set of sy mpt oms k D and the symptom sk dD  can be presented in the following form: ( , ) ( ) ( / , ) B k s B B k s J D d H E H E D d  (81) After substituting (54) and (76) into (81) and t ransformation with the (75) t aken into account, the following is arrived at: 1 1 00 ( , ) ( 1 ) k m B k s jl jl jl J D d n p       (82) What results from the above-considered issues will be used to prove the Lemma 7, and Theorem 2. Lemma 7 The total information of the set of s ymptoms k DD  and the symptom sk dD  is equal to the sum of the information of the set k D and conditional information of the symptom s d ( , ) ( ) ( / ) B k s B k B s k J D d J D J d D  (83) Proof (see APPENDIX) Theorem 2 The information of the set of sy mptoms { }, 1 , ... , , K k K D d k K D D    equals the sum of conditional information of individual symptoms. ( ) 1 1 ( ) ( / ) K B K B k k k J D J d D     (84) Proof (draft): The proof is carried out with the induction method. 16 For 1 K  L(84)= 1 ( 1) ( ) ( ) BB J D J d  R(84)= ( 1 ) 0 (1 ) ( / ) ( ) BB J d D J d  that is L(84) = R(84) Assuming that (84) proves true for any 1 ( ) M c a rd D  ( ) 1 1 ( ) ( / ) M B M B k k k J D J d D     (85) After having added ( 1) ( / ) B M M J d D  to both sides of (85), the following is arrived at: ( 1) ( ) 1 1 ( 1) ( ) ( / ) ( / ) ( / ) M B M B M M B k k k B M M J D J d D J d D J d D         (86) Using th e Lemma 7, eq u ation (86) can be brought to the following form (a fter the transformation of the right side thereof): 1 1 ( ) 1 1 ( ) ( / ) M B M B k k k J D J d D      This, according to the principle of induction, makes the proof of the theorem complete. The proof has shown that the combinatoria l-probabilistic diagnostic information has the property of additivi ty. I f the amount of infor mation delivered b y set of symptoms equals to the entropy value then system condition is identified with necessary accur ac y . APPENDIX Proof of Lemma 1 (draft): After simple trans formations on the left side of (43), and changes in summation indices, the following is arrived at: L (43) 1 ( )( 1 ) n i i P e n    (87) The comparison between (43) and (87) shows that: L (43) = R (43) q.e.d. Proof of Lemma 2 (draft): On the basis of (44), the following can be written, respectively: 1 1 ) ( ) ( )( 1 ) ) ( ' ) ( ' )( ' 1 ) n Bi i n Bi i a H E P e n b H E P e n       (88) After subtracting the sides of equations (88)a and (88)b, and after the transformation with assumption (41) taken into account, the following is arrived at: ( ) ( ' ) ' BB H E H E n n    (89) The following equivalence directly results from the relationship (89): 17 ( ) ( ' ) ' BB H E H E n n    q.e.d. Proof of Lemma 3 (draft): After substituting 1 n  into (44) the following is arrived at: 1 ( ) 0 Bn HE   q.e.d. Proof of Lemma 4 (draft): On the gr ounds of the assumptions (45) and (46) , the left side of (47) after suitable transformations, is arrived at: L (47) ( 1 ) jj pn  (90) The comparison between (47) and (90) shows that: L (47)= R (47) q.e.d. Proof of Lemma 5 (draft): The average uncertainty of the condition, for some assumed pa rtition, can be found from the definition of the expected value: * 11 ( / { }) ( / ) ( 1 ) mm B j j B j j j jj H E E p H E e E p n       (91) On the other hand, on the grounds of the Lemma 4, after the summation as re lated to all j , the following is arrived at: 11 ( ) ( 1 ) mm B j j j jj H E p n    (92) The comparison between (91) and (92) shows that: 1 ( / { }) ( ) m B j B j j H E E H E    q.e.d. Proof of Lemma 6 (draft): With the Lemma 5 employed, the unc ertainty of the condition can be written in this case in the following form: 1 ( / {{ }}) ( 1 ) n B i j j j H E e p n    (93) After substituting 1 , 1 , . . . , j n j n  into (93), the following is arrived at: ( / {{ } } ) 0 Bi H E e  q.e.d. Proof of Lemma 7 (draft): With (80) and (78) as the basis, the following can be written, respectively: 1 0 1 1 00 ) ( ) ( 1 ) ) ( / ) ( ) k k m B k j j j m B s k jl j jl jl a J D n p b J d D n p p           (94) 18 Having summed up (94)a and (94)b by sides, 1 0 11 1 1 1 0 0 0 0 0 ( ) ( / ) ( 1 ) () k kk m B k B s k j j j mm jl j jl jl jl jl j l j l l J D J d D n p n p p n n p                             is obtained. Finally, 1 1 00 ( ) ( / ) ( 1 ) k m B k B s k jl jl jl J D J d D n p         (95) What results from the comparison between (82) and (95) is ( , ) ( ) ( / ) B k s B k B s k J D d J D J d D  q.e.d. REFERENCES [1] M. Blanke, M. Kinnaert, J. Lunze, M. Staroswiecki, J. Schröder, Diagnosis And Fault - Tolerant Control , Berlin: Springer, 2003. [2] H. Borowczyk, Quasi -Information Method Of Complex Technical Objects Diagnostic Algorithm Assessment . (P h.D. Diss.), Warsaw: M ilitary Universit y of T echnology, 1984 (in Polish) [3]   Artificial Intelligence, 1987, vol. 32 pp. 97-130 [4]     -Based Fault Detection And Diagnosis     Ann. Rev. in Control , 2004, pp 71 -85 [5]            Fault D iagnosis. Models, Artificial Intelligence, Applications . Berlin: Springer-Verlag 2004. [6] M. S yfert, The Issue Of Diag nostic Relation Uncertainty And Fault Conditional I solability. In 6th IFAC Symp. Fault detection, supervision and safety of technical processes, IFAC Proceedings Volumes , Amsterdam: Elsevier, 2007, pp. 793-798 [7]                      6th IFA C Symp. Fault detection, supervision and saf ety of technical processes, IFAC Proceedings Volumes , Amsterdam: Elsevier, 2007, pp. 679-684 [8]       Multi-Level Method Of T echnical Objects' C ondition Evaluation And Diagnostic System Synthesis . D. Sc. Diss., Bull. Of Mi litary Universit y of Technology, No. 8 396, Warsaw: MUT Publ., 1985 (in Polish) [9] J . A. Starzy k, Dong Liu, Zhi-Hong Liu, D. E. Nelson, J .    -Based           IEE E Transactions On Instrumentation And Measurement , Vol. 53, No. 3, June 2004 [10] R. Sui , F. Tu, K. R. Pattipati, A. Patterson -       uencing  IEEE Trans. On Systems, Man, and Cybernetics - Part B, Vol. 34. No. 3, June 2004, pp. 1490-1499 [11]             IEEE Trans. On Systems, Man, and Cybernetics Part A. vol. 33, pp. 86-99. Jan. 2003. [12]           -Diagnosis Algorithms for All-          IEEE Journal Of L ightwave Technology . Vol. 23. No. 10. October 2005 19 [13] P. M. Frank, E. Alc orta Garcia, B. Koppen-         Math. Comp. in Simulation , 2000 pp 259  271 [14]             -Based Parametric Diagnostics Of Transport-Dedicat      Aircraft Engineering and Aerospace Technology , Vol.77, 3, 2005, pp. 222  227. [15]             Math. Comp. in Simulation , 1998pp. 465  483 [16]     T echnical Objects' Diagnostics . R adom: ITE Publ., 2002 (in Polish) [17]                 Automatica , 1995, 315, 747-753. [18]  -Logarithmic Probabilistic Rates of the Object Condit ion's Uncertainty          6th IFAC Sy mp. Fault detection, supervision and safety of technical pro cesses, IFAC Proc eedings Volumes , Elsevier Amsterdam, 2007, pp. 493  498 [19] M. H. L   -               14th Int. Workshop on qualitative reasoning , Morelia, Mexico 2000, 89  96. [20]                 Theory     IEEE Tr ans. On Systems, Man, and Cybernetics , vol. SMC-20, pp. 872-887. Jul y /Aug. 1990. [21]         Automatica 1996 Vol. 32, 3, pp. 385-390 [22] P.K. Va rshney, C.R.P. Hartman n, J.M. De            IEEE T rans, on Compute rs . Vol. C-31,No. 2,1982, pp. 164- 170. [23] M. Basseville               Automatica , Vol. 33 No. 5, 1997, pp. 783-603 [24]            -Diagnosis for All -Optical                -14, 2006, 2919-2923 [25]            The Bell Technical Journal , Vol. 27, 1948, pp 379  423, 623  656 [26] J.            Proc. 7th Prague Conference on Information Theory, Statistical Decision Functions and Random Processes , 1974, pp.73-86 [27]          Colloquium Mathematicum . Vol IX Fasc 1, Warsaw Poland, 1962, pp 131-150. [28] A.            Pro c. Fourth Berkeley Symp. Math. Stat. Prob. Vol I p 547. Univ. of California Press, Berkeley, 1961 [29]               Structural Alpha-  Kybernetica , vol.3, 1967, pp.30-35. [30]         M easurable Partitions. Probability and Information Theory ” , Lectur e Notes in Mathematics , vol. 296, Springer- Verlag, 1973, pp. 102-138 [31] J. Aczél, Z. Daròcz y,. On Measures of Information and Their Characterizations . Academic Press, Mathematics in Science a nd Engineering, 1975 [32] B. Eb anks, P. Sahoo, W. Sander, Characteriz ations of information measures . World Scientific, 1998 20 [33]             IEEE Trans. On Information Theory , vol. 48, no 7 Jul y 2002, p p. 2138-2142 [34] D. H. Stamatis, Failure Mode and Effect Analysi s, FMEA from Theory to Execution . ASQ Quality Press, 2003. [35]                IEEE Trans. On Systems, Man, and Cybernetics, Part A , No. 282, 1998, pp. 221-234. [36] Z. Pawlak,. Rough Sets. T heoretical Aspects of R easoning about Data . Kluwer Ac ademic Publishers, 1991 [37]       IEEE Tr ans. On Systems, Man, and Cybernetics , vol. 24, no 7, Jul y 1994, pp. 1074-1082 [38] P. P. P arkhomienko, E. S. Sogomonian, Foundations of Technical Diagnostics , Energoizdat, Moskva 1980 (in Russian) [39]           -Optimal Test Sequencing       IEEE T rans. On Systems, Man, and Cybernetics Part A , vol. 29, Jan. 1999, pp. 11-26.