State and history in operating systems

State and history in o p erating systems Victor Y o daik en Cop yrigh t 2008.  yodaiken@finitestateresearch.com No v em b er 23, 2018 1 Metho ds and application to op er ating systems The classic UNIX co de for switc hing pr o cesses is famo usl y opaque and co n- cise. In V ersion 6 of UNIX[ 5 ], Dennis Ritc hie app ended a half-hearted ex- planation and then added a wry: Y ou are not exp ected to understand this. Suc h complex state c hanges are at t he heart of OS design. In t hi s note, I will sp ecify what the co de do es and, I hop e, illustrate metho ds that will b e of reaso nably gene r a l utilit y in understanding and designing complex computer and soft w are systems. The co de itself lo oks something lik e this. 0 s w i t c h ( p r o c e s s _ t  n e x t ) { 1 i f ( s a v e ( ) ) { 2 r e s u m e ( n e x t ) ; 3 p a n i c ( " r e t u r n e d f r o m r e s u m e " ) ; 4 } e l s e f i x m m u ( ) ; / / s w i t c h i n g i n 7 r e t u r n ; Sa v e sa v es the pro cess state of the curren t (running) pro cess and re- turns "1" so that the runni ng pro cess then calls "resume" with a p oin ter to the sa v ed pro cess state of a second pro cess  "next". The "resume" subroutine restores the st ate of the pro cess iden tied b y "next" and returns  P ermissio n gran ted to mak e and distribute complete copies for non-commercial use but not for u se in a publication. All other righ ts reserv ed but fair use encouraged as long as prop erly cited. 1 "0" as if it wer e r eturning fr om save . The newly restored pro cess then falls through to the co de section mark ed "switc hing in". The sa v ed pro cess do es not start running again un til some other pro ce ss calls "resume" with a p oin te r to its data structure. P r ocess p 1 P r ocess p 2 P r ocess p k sav e p 1 state r etur n 1 to " if " cal l r esume ( p 2 ) r etur n 0 to " if " enter el se f ixmmu r etur n ::: sw itchout ::: sav e p k state cal l r esume ( p 1 ) r etur n 0 to " if " There is no time limit b et w een a pro cess sa ving and resuming and the system ca n get up to an y n um b er of things in-b e t w een the t w o op erat ions  ev en susp ending itself. When pro cess p 1 calls "switc h" with next = p 2 , then p 1 will not get to "return" un til p 2 has resumed op eration and then, in some future state, some other pr o cess p k where p k ma y o r ma y not b e the same as p 2 calls switc h with next = p 1 . One w a y to express this prop ert y is to sa y that an y path z that starts in pro ce ss p 1 at the start of "switc h" and that terminates in pro cess p 1 at the return from "switc h" m ust b e factorable i n to subpaths that v is it a series of in termediate states - as sho wn i n this diagram. z u1 u3 u2 u4 next=B (A,switch,call resume) (B,switch,fixmmu) (C,switch, call resume) next=A (A, switch, enter) (A,switch, return) 2 The diagram is suggestiv e, but it w o uld b e nice to b e able to write do wn exactly what it means and then see ho w othe r prop erties in teract with this prop ert y or what else w e ha v e to kno w ab out the system to assure this prop ert y . Let w b e the "curren t ev en t sequence"  the path t hat leads from the initial state of the system t o the curren t state 1 . Let  b e the empt y sequenc e, w a b e the sequence obtained b y app ending ev en t a to sequence w , and w  z the sequence obt ained b y app ending sequence z to sequence w . Then  leads to the initial state. App ending an ev en t driv es the system to a successor state from the curren t sta te. App ending a sequence of ev en ts driv es the system to a future state. Recursiv e relations are sucien t to dene man y ev en t sequence dep enden t v ariables. Here's a tri v ial one that just coun ts all ev en ts. C ount (  ) = 0 and C ount ( w a ) = C ount ( w ) + 1 In what follo ws, I will assume the existence of a collection of sequence dep enden t v ariables and f u nctions that pro vide a windo w in to state. De - nitions of some of those f u nctions from simpler state v ariables a r e giv en in section 3 . Supp ose w e ha v e functions Cline and CName so that Cline ( p; w ) and CName ( p:w ) are, resp e ctiv ely , the curren t line n um b er and curren t function name of pro cess p in the source co de listing (assumed here to b e in "C"). Then a "debugger" view of system state is giv en b y: Lo c( w ; p ) = ( CF unc ( w ; p ) ; Cline ( w ; p )) F or ev ery progra m v ariable x , w e let V ( w ; p; x ) b e the curren t v alue of x in the con text of pro cess p . F or ex ample, when p is inside "switc h" the v alue o f V ( w ; p; next ) is the pro cess iden tier of the target of the switc h. Note that f ( w  z ; g ( w )) ev aluates g in the state reac hed b y w and ev aluates f in the state reac hed b y w  u . So Lo c( w  u; V ( w ; p; next )) is the lo cation of pro cess p 0 = V ( w ; p; next ) in the state determined b y w  u . Prop osition 1.1. If Lo c( w ; p 1 ) = ( sw itch; 0) and V ( w ; p 1 ; next ) = p 2 6 = p 1 then for any z so that Lo c( w  z ; p 1 ) = ( sw itch; 7) ther e must b e a pr o c ess 1 There is a common theory that w e ha v e to pretend computational ob jects are "n on- deterministic", b ut that seems to b e based on mistaking metho dological limitations for fundamen tal prop erties. 3 p k 6 = p 1 and se quenc es ( u 1  u 2  u 3  u 4 ) = z so that: Lo c( w  u 1 ; p ) = ( sw itch; 2) (1) Lo c( w  u 1  u 2 ; p 2 ) = ( sw itch; 4) (2) Lo c( w  u 1  u 2  u 3 ; p k ) = ( sw itch; 0) (3) V ( w  u 1  u 2  u 3 ; p k ; next ) = p 1 (4) Let's supp ose w e ha v e a b o olean function "Running" so that R unning ( w ; p ) = 1 if and only if p is activ e.  R unning ( w ; p ) 2 f 0 ; 1 g ;  R unning ( w ; p ) > 0 if and only if p is running in the state determined b y w ;  R unning ( ; p ) > 0 if and only if p is running in the initial state of the system;  R unning ( w ; p ) > R unning ( w a; p ) if and only if ev en t a causes p to stop running if the syst em is in the state det ermined b y w ;  there is a prex u of z so that R unning ( w  u; p ) if and only if p is "sometimes" running during z after the state determined b y w . By using ev en t sequences w e get an activ e view of ho w v ariables c hange and it is easy to dene v ariables t hat help rev eal the w orkings o f a system. Here's one that coun ts the n um b er of times a pro cess has "switc hed in". I n ( ; p ) = 0 ; I n ( w a; p ) = ( 1 + I n ( w ; p ) If R unning ( w ; p ) < R unning ( w a; p ) I n ( w ; p ) other w is e One of the adv an tages o f the metho ds used here is that w e are not forced to either en umerate the state set or ev en explain to o m uc h ab out the alphab et of ev en ts. F or something lik e an OS, the ev en t alphab et is going to b e large and complex and the state se t will b e w orse. P erhaps the ev en t alphab et will consist of "samples" of the inputs applied t o the c hips of the motherb oard at eac h pro cessor cycle. W e could imagine these ev en ts as digitized snapshots of signals. Eac h snapshot then indicates some discrete in terv al of time has passed. T he r e ma y also b e ev en ts that corresp ond to 4 logical c hanges. But, for no w, w e can just sp ecify the information w e need to b e able to dec o de from the ev en t strea m. Let's require that line n um b ers and source co de functions only c hange when a pro cess is activ e. Lo c( w a; p ) 6 = Lo c( w ; p ) = ) R unning ( w ; p ) Note that R unning ( w ; p ) = 1 ma y not mean R unning ( w ; p 0 ) = 0 b e- cause w e lea v e op en the p ossibilit y of m ultiple pro cessor co r e s. More on that b elo w. Note that V ( w a; p; x ) 6 = V ( w ; p; x ) do es not nece ss a r ily imply that R unning ( w ; p ) = 1  b eca use man y of the ob jects within the address space of a pro c ess are shared ob jects. F or example the pages ma y page in or out, data ma y arriv e from a DMA devic e, there ma y b e notication of an I/O or other e v en t, a nd shared data structures will b e mo died b y other pro cesses. Mo dularit y in op erating systems is a to ugh engineering c hallenge. 2 Instrumen ting the OS Prop osition 1.1 is a "safet y" prop ert y  it requires that if there is a path from en try to exit, the path m ust ha v e certain prop erties. W e also need a liv eness prop ert y  that pro cesses will adv anc e from switc h to the running of the target pro cess. If eac h ev en t denes signa l s o v er a sp ecied unit of time, then w e can ha v e Time ( w ) pro vide the curren t time in some sucien tly ne unit. Without going in to to m uc h deta i l, Time needs to b eha v e sensibly: Time ( w )  Time ( w  u ) W e will ofte n need to coun t ho w m uc h time passes during an ev en t or se- quence of ev en t s Time ( w  u )  Time ( w ) tells us ho w m uc h time passes during u after w and Time ( w a )  Time ( w ) measures the time during the s ing l e ev en t a . It ma y b e that the r e are ev en ts that tak e no real-time or ma yb e eac h ev en t corresp onds t o a sample 5 of signals during a discrete in terv al or ev en that ev en t duration dep ends on history . W e don't ha v e to w orry ab out an y of that y et. Let's also supp ose w e ha v e V alidPr o c ess ( w ; p ) to tell us whether a pro- cess iden tier p iden ties an act ual, instan tiated pro cess (on an y core) and w e ha v e R eady ( w ; p ) 2 f 0 ; 1 g to te l l us is a pro cess is ready to run. V alidPr o c ess ( w ; p ) 2 f 0 ; 1 g R e ady ( w ; p ) 2 f 0 ; 1 g R unning ( w ; p )  V alidPr o c ess ( w ; p ) R e ady ( w ; p )  V alidPr o c ess ( w ; p ) W e can no w dene ho w long a pro cess has b een w aiting to run. Waiting ( ; p ) = 0 Waiting ( w a; p ) = ( ( Time ( w a )  Time ( w )) + Waiting ( w ; p ) if R unning ( w ; p ) < R e ady ( w ; p ) 0 other w ise A a system i s t liv e liv e if Waiting ( w ; p ) < t liv e for all w . Although some researc hers ha v e decided that "liv eness" should b e considered a prop ert y "in the limit" (without a n explicit time b ound), I don't think suc h a v ersion of liv ene ss means an ything in teresting when w e are discussing engineered discrete sta te ob jects. Prop osition 2.1. Cal ling switch for c es pr o c ess "next" to run within a xe d time. Ther e is a t sw itch so that for any w and z : If Lo c( w ; p ) = ( sw itch; 0) and Time ( w  z )  Time ( w ) + t sw itch then ther e is a pr ex u of z so that Lo c( w  u; V ( w ; p; next )) = ( sw itch; 5) Prop osition 2.1 has to b e true i f the syste m is t liv e liv e. Otherwise, t he switc hing out pro cess could stall, forev er. The t w o prop ositions formalize what w e w an t the switc h co de to do at a high lev el, but do not sp ecify ho w state m ust b e preserv ed o v er a switc h. Since pro cess sta te consists of b oth shared and non- s ha r e d data, w e ha v e to distinguish those: 6 Prop osition 2.2. If Lo c( w ; p ) = ( sw itch; 3) and Lo c( w  u; p ) = ( sw itch; 5) and ther e is no pr op er pr ex z of u so that Lo c( w  z ; p ) = ( sw itch; 5) then for any non-shar e d variable x; V ( w ; p; x ) = V ( w  u; p; x ) 3 Digging do wn Here's a list of functions " assumed" in t o existence ab o v e that need to b e either justied or dened fr o m simpler elemen ts. Cline Cname Save dR e gisters StackContents V alidPr o c ess R e ady R unning Time NonShar e d V Let's supp ose that the mac hine has 1 or more cores and that Reg ( w ; c; r ) ; Mem( w ; c; l oc ) are, resp ectiv ely the con ten ts o f register r on core c and the con ten ts of mem- ory lo cation l oc on core c . F or example Reg ( w ; c; P C ) (program coun ter) and Reg ( w ; c; S P ) (stac k p oin ter) are useful to kno w. Giv en a program list- ing L and the curren t program c oun ter, it is reasonably straigh tforw a r d t o compute CLine and CName , so I w on't dig in to those further. Giv en these v alues, whether a sym b ol is a stac k or globa l v ariable is also straigh tforw ard, so w e assume IsStack and IsGlob al can b e constructed. F urthermore, for global v ariables the corresp ondence b et w ee n name and a ddr e ss is determined b y the program listing a nd some data ab out the compiler/link er settings. Supp ose there is a memory lo cation cur r ent [ c ] for ea c h core c that holds the iden tit y of the curren t pro cess on core c . Then Mem( w ; c; cur r ent [ c ]) is the pro cess running on core c . W e ha v e t o require that Mem( w ; c; cur r ent [ c ]) = Mem( w ; c 0 ; cur r ent [ c 0 ]) $ c = c 0 7 and then R unning ( w ; p ) ( 1 if for some c; Mem( w ; c; cur r ent [ c ]) = p 0 otherwise. R e ady ( w ; p ) 8 > < > : 1 if for an y c; B itset (Mem( w ; p + pr ocstatus ) ; R E AD Y ) and V alidPr o c ess ( w ; p ) 0 otherwise. V ( w a; p; x ) 8 > > > < > > > : Mem( w a; c; y ) if Mem( w ; c; cur r ent [ c ]) = p and I s Gl obal ( w ; p; x ) and y = x and I s S tack ( w ; p:x ) and y = x + Reg ( w ; c; S P ) V ( w ; p; x ) otherwise. If Reg ( w ; c; S P ) is the con ten ts o f the stac k p oin ter registe r on core c , then Mem( w ; c; Reg ( w ; c; S P )) is the con ten ts of the top of the stac k on pro cessor c ore c (assuming alignmen t and so on). I n man y op erating systems, the k ernel stac k of a pro cess, whic h is what w e are discussing here, is xed size and "gro ws do wn" b y s ubt r a ction from a , for example, 8K b oundary . One of the rea sons for doing this is that its easy to calculate the stac k base b y bitw iseand ( stack a d dr ess + 8095 ; bitinv er t (8095)) if the stac k is 8K and on an 8K b oundary . In that case, w e can dene S t ac kCon ten ts so it capt ur e s the stac k. StackContents ( w a; p ) = 8 > > > > > > > > < > > > > > > > > : (Mem( w ; c; a ) ::: Mem( w ; c; b )) if R unning ( w ; p ) and Mem( w ; c; cur r ent [ c ]) = p and a = Reg ( w ; c; S P ) and b = bitiseand ( a + 8195) ; bitinv er t (8195)) and increme n ts b et w ee n a and b are b y w or ds iz e StackSize ( w ; p ) otherwise Note that Stac kCon ten ts is dened so that it do es not c hange when t he pro cess is not running. If w e dig do w n to the assem bler lev el, w e'd probably w an t to b e sure that the stac k con ten ts at the p oin t o f return from sa v e w as the sam e as that at the p oin t of return from resume. 8 Cline from Reg ( w ; c; P C ) Cname from Reg ( w ; c; P C ) Save dR e gisters from Reg ( w ; c; :: ) StackContents from Reg ( w ; c; S P ) and Mem( w ; c; :: ) V alidPr o c ess from Mem( w ; c; p  > status ) R e ady from Mem( w ; c; p  > status ) R unning from Reg ( w ; c; cur r ent ) Time primitiv e NonShar e d from sym b ol table V from Mem( w ; c; ::: ) 4 P arallelism and encapsulation P arallelism is a h uge issue in "formal metho ds" but app ears naturally here. F or example, it is certainly p ossible that for some w and a there are sev eral cores c so that Reg ( w a; c; P C ) 6 = Reg ( w ; c; P C ) . W e ha v e not had to y et sp ecify a n ything a b out the w a y the cores c ha nge state in parallel  they just a r e sp ecied in a w a y that mak es it p ossible. In some cases, ho w ev er, w e w an t t o describ e systems in whic h the arc hitecture of comp onen ts is sp ecied and that is also straigh tforw ard. Consider an abstract mo del of pro cess in teraction where pr o cesses can either w ait for or generate ev en ts and, only one pro cess can adv ance p er core. W e are g oing to w an t to connect up a collection of these pro cesses so that they comm unica te syc hronously . step receive send step output=running output=sending output=waiting Note that the diagram obscures the in te n t that there ma y b e man y dieren t states where output is running, w aiting, or sending. Denition 4.1. f is an abstr act state pr o c ess over P and X with id p 0 9 if and only if f ( w ) 2 f r unning ; w aiting [ p ] ; sending [ x; p ] : x 2 X ; p 2 P g and f ( w ) 6 = r unning = ) f ( w  h step i ) = f ( w ) and f ( w ) = sending [ x; p ] = ) f ( w  h send i ) = idl e and f ( w ) = w aiting [ p ] = ) f ( w  h r eceiv e [ x; p ] i ) = idl e and f ( w ) 6 = w aiting [ p 0 ]  never wait for self and f ( w ) 6 = sending [ x; p 0 ]  never sent to self Man y distinct sequence dep enden t functions can satisfy this sp ecica- tion. Tha t is, w e can ha v e A 1 and A 2 that are b oth abstract pro cesses b y this denition where A 1 ( w ) 6 = A 2 ( w ) for some o r ev en most w . An abstract pro cess that is "running" has some in ternal pro cedure for deciding when to request to send or receiv e a message. W e do not need, no w, to decide what that pro cess is, but it could easily b e the execution of a program it receiv es as a message or something xed in its in ternal op eration or some com bination. Finally , w e ha v e not sp e cied what happ ens when un w an ted ev en ts happ en  suc h as a receiv e fr o m p 0 when the pro cess w an ts to receiv e from p . No w let's dene a connected system of suc h abstract pro cesses. Supp ose that e ac h of A p 1 : : : A p k are abstra ct pro cesses and dene F ( w ; p ) = A p ( w p ) where w e will dene w p recursiv ely .  p =  and ( w a ) p = w p  g ( w ; a; p ) and g ( w ; a; p ) = 8 > > > > > > > > > > > < > > > > > > > > > > > : h r eceiv e [ x; q ] i if A p ( w p ) = w aiting [ q ] and A q ( w q ) = sending [ x; p ] h send i if A ( w p ) = sending [ x; q ] and A ( w q ) = w aiting [ p ] h step i if A ( w p ) = r unning and R unning ( w ; p )  otherwise. Note that p only g ets to "step" if it is select ed as t he running pro cess in the enc ompassing en vironmen t of the op erating system. 10 5 Conclusion and mathematical note In brief, sequence f unc tions are represen tations of Mo ore t yp e state ma- c hines. Giv en a sequence function f o v er alphab et B let B  b e the set of nite seque nces o v e r B including  and dene w  f z ( ) 8 u 2 B  ; f ( w  u ) = f ( z  u ) Then dene [ w ] f = f z : z f w g and consider the set of these equiv alence classes S f = f [ w ] f : w 2 B  g . Dene  f ([ w ] f ; a ) = [ w a ] f and dene  f ([ w ] f ) = f ( w ) . Then M f = ( B ; S f ; [  ] f ;  f ;  f ) is a classical (although not necessarily nite) Mo ore mac hine with state set S f , initia l state [  ] f , transition ma p  f , and output map  f . Con v e r se ly , giv e n a Mo ore mac hine M = ( B ; s 0 ;  ;  ) dene f M so that f M ( w ) =  (   ( w )) where   (  ) = s 0 and   ( w a ) =  (   ( w ) ; a ) . The encapsulatio n of section 4 corresp onds to a Mo ore mac hine pro- duce called the gener al pr o duct [ 2 ]. F or simplicit y let's dene this pro d- uct for nite n um b ers of sta te mac hines. Supp ose f : B   X ! Y where X = f x 1 ; : : : x k g is dened b y f ( w ; x ) = g ( w x ) where  x =  and ( w a ) x = w x   ( f ( w ; x 1 ) : : : ; f ( w ; x k ) ; a; x ) . F or ev en more simplic- it y , supp ose  ( y 1 ; : : : y k ; a; x i ) 2 B i . Then for eac h i w e can construct a M g i = ( S i ; s 0 i ;  i ;  i ) using t he construction ab o v e. Dene a pro duct b y M f = ( i B i ; ( s 0 0 :::s 0 k ) ;  ;  ) . Eac h state of M f is a k -tuple s = ( s 1 ; : : : s k ) 2  i S i . The transition function  is constructed as follo ws:  ( s; a ) = (  1 ( s 1 ;  (  1 ( s 1 ) : : :  k ( s k ) ; a; x 1 )) ; : : :  k ( s k ;  (  1 ( s 1 ) : : :  k ( s k ) ; a; x k ))) : Finally:  (( s 1 ; : : : s k )) = (  1 ( s 1 ) ; : : :  k ( s k )) . Then f M f ( w ) = ( f ( w ; x 1 ) : : : f ( w ; x k ) . It ma y b e seen wh y the functional represen tation is a dv an tageo us in some situations. Consideration o f the algebraic basis of state mac hine theory and the re- lationship b et w een state mac hines a nd semigroups indicates that there ma y b e some v alue in lo oking at the algebraic structure of sequence dep enden t functions. If  = f is dened so that w  = f u ( ) 8 z 1 ; z 2 ; f ( z 1  w  z 2 ) = f ( z 1  u  z 2 ) then the congruence classes [[ w ]] f = f u : w  = f w g form a monoid under the op eration [[ w ]] f  [[ u ]] f = [[ w  = u ]] f . If w e constrain  to not dep end on an y 11 feedbac k, so that transitions to M i dep end only on outputs of M j : j < i , then the results of Krohn-Rho des theory a s describ ed in Ho l c om b e [ 4 ], Arbib [ 1 ] and Ginzburg [ 3 ]. What happ ens if  is constrained in other w a ys, suc h as b y a certain c ir c ui t design discipline? Also, in databases, using som e circuit disciplines, and in other situations, in v ertibilit y is a useful prop ert y . That in v ertibilit y pro duces sequence functions that corresp ond to groups. A m uc h earlier v ersion of this w ork can b e found in [ 9 ] and [ 8 ] and m uc h earlier in [ 7 ] with applicatio ns in [ 6 ] and [ 10 ]. Unfortunately , it to ok me man y y ears to understand go o d advice from Professor George A vrunin that the forma l logic notation w as an imp edimen t instead of an adv an tage. References [1] Mic hael A. Arbib. A lgebr aic the ory of machines, langu ages, and semi-gr oups . A cademic Press, 1 968. [2] F erenc Gecseg. Pr o ducts of A utomata . Monographs in Theoretical Computer Science. Springer V erlag, 1986. [3] A. Ginzburg. A lgebr aic the ory of automata . A cademic Press, 1968. [4] W.M.L. Holcom b e. A lgebr aic A utomata The ory . Cam bridge Univ er- sit y Press, 1983. [5] John Lions. Lions' c ommentary on UNIX 6th e dition with sour c e c o de . P eer-to-P eer C o mm unications, Inc., San Jose, C A, USA, 1996. [6] V. Y o daik en and K. Ramamritham. V erication of a reliable broadcast algorithm. In J. V ytopil, editor, F ormal T e chniques in R e al-Time and F ault-T oler ant Systems , n um b er 57 1 in LNCS. Springer-V erlag, 1992. [7] Victor Y o daik en. Mo dal functions for conc i se represen tation of nite automata. Information Pr o c essing L etters , No v 20 1991. [8] Victor Y o daik en. Discrete stat e v ariables. T ec hnical Rep ort h ttp://www.y o daik en.com/pap ers/s8.p df, 2005. [9] Victor Y o daik en. Discrete stat e v ariables. T ec hnical Rep ort h ttp://www.y o daik en.com/pap ers/s10.p df, 2006. 12 [10] Victor Y o daik en a nd Krithi Ramamritham. Sp ecica tion and v erica- tion of a real-time queue using mo dal algebra. In IEEE R e al Time Systems Sym p osium , 1990 . 13