Fork Sequential Consistency is Blocking

F ork Sequential Consistency is Blocking Christian Cachin ∗ Idit K eidar † Alexander Shraer † July 28, 2021 Abstract W e consider an untrusted server storing shared data on behalf of clients. W e show that no storage access protocol can on the one hand preserve sequential consistenc y and wait-freedom when the server is correct, and on the other hand alw ays preserve fork sequential consistency . 1 Intr oduction W e examine an online collaboration facility providing storage and data sharing functions for remote clients that do not communicate directly [3, 4, 13, 14]. Specifically , we consider a server that implements single-writer multi-reader registers. The storage server may be faulty , potentially e xhibiting Byzantine faults [10, 8, 11, 2]. When the server is correct, strong liv eness, namely wait-freedom [5], should be guaranteed, as a client editing a document does not want to be dependent on another client, which could e ven be in a different timezone [14]. In addition, although read/write operations of different clients may occur concurrently , consistency of the shared data should be provided. Specifically , we consider a service that, when the serv er is correct, pro vides sequential consistency , which ensures that clients ha ve the same view of the order of read/write operations, which also respects the local order of operations occurring at each client [7]. Sequential consistency provides clients with a conv enient abstraction of a shared storage space. It allows for more ef ficient implementations than stronger consistency conditions such as linearizability [6], especially when the system is not synchronized [1]. In ex ecutions where the server is faulty , liveness ob viously cannot be guaranteed. Moreov er , with a Byzantine serv er , ensuring sequential consistenc y is also impossible [2]. Still, it is possible to guarantee weaker semantics, in particular so-called forking consistency notions [8, 10]. These ensure that when- e ver the serv er causes the vie ws of tw o clients to dif fer in a single operation, the tw o clients ne ver again see each other’ s updates after that. In other words, if an operation appears in the views of two clients, these vie ws are identical up to this operation. Originally , fork-linearizability was considered [8, 10, 2]. In this paper , we examine the weaker fork sequential consistency condition, recently introduced by Oprea and Reiter [11], who showed that this ne w condition is sufficient for certain applications. Howe ver , to date, no fork-sequentially-consistent storage protocol has been proposed. In fact, Oprea and Reiter suggested this as a future research direc- tion [11]. Furthermore, Cachin et al. [2] sho wed that the stronger notion of fork-linearizability does not allo w for wait-free implementations, but conjectured that such implementations might be possible with fork sequential consistenc y . Surprisingly , we prove here that no storage access protocol can provide fork sequential consistenc y at all times and also be sequentially consistent and w ait-free whene ver the server is correct. This generalizes the impossibility result of Cachin et al. [2], and requires a more elaborate proof. ∗ IBM Research, Zurich Research Laboratory , CH-8803 R ¨ uschlikon, Switzerland. cca@zurich.ibm.com † Department of Electrical Engineering, T echnion, Haifa 32000, Israel. { idish@ee, shralex@tx } .technion.ac.il 1 In this paper we require only sequentially consistent semantics when the server is correct. Though one may also consider stronger semantics, such as linearizability , for this case, as our goal is to prove an impossibility result, it suffices to address sequential consistency . Our impossibility result a fortiori rules out the existence of protocols with stronger semantics as well. 2 Definitions System model. W e consider an asynchronous distributed system consisting of n clients C 1 , . . . , C n , a server S , and asynchronous FIFO reliable channels between the clients and S (there is no direct communication between clients). The clients and the server are collectiv ely called parties . System components are modeled as deterministic I/O Automata [9]. An automaton has a state, which changes according to transitions that are triggered by actions . A pr otocol P specifies the behaviors of all parties. An ex ecution of P is a sequence of alternating states and actions, such that state transitions occur according to the specification of system components. All clients follow the protocol, and any number of clients can fail by crashing. The server might be faulty and deviate arbitrarily from the protocol, exhibiting so-called “Byzantine” faults [12]. A party that does not fail in an execution is correct . The protocol emulates a shar ed functionality F to the clients, defined analogously to shared-memory objects. Events, operations, and histories. Clients interact with the functionality F via operations provided by F . As operations take time, they are represented by two events occurring at the client, an in vocation and a r esponse . An operation is complete if it has a response. For a sequence of ev ents σ , complete ( σ ) is the maximal subsequence of σ consisting only of complete operations. A history is a sequence of requests and responses of F occurring in an execution. An operation o pr ecedes another operation o 0 in a sequence of e vents σ , denoted o < σ o 0 , whene ver o completes before o 0 is in v oked in σ . T wo operations are concurr ent if neither one of them precedes the other . A sequence of ev ents is sequential if it does not contain concurrent operations. A sequence of ev ents π pr eserves the r eal-time order of a history σ if for every two operations o and o 0 in π , if o < σ o 0 then o < π o 0 . For a sequence of ev ents π , the subsequence of π consisting of ev ents occurring at client C i is denoted by π | C i . For a sequential π , the prefix of π ending with operation o is denoted by π o . An execution is admissible if the following two conditions hold: (1) the sequence of ev ents at each client consists of alternating in vocations and matching responses, starting with an in vocation; and (2) the ex ecution is fair . F airness means, informally , that the execution does not halt prematurely when there are still steps to be taken or messages to be deli vered (we refer to the standard literature for a formal definition of admissibility and fairness [9]). Read/write registers. A functionality F is defined via a sequential specification , which indicates the behavior of F in sequential ex ecutions. The basic functionality we consider is a r ead/write r e gister X . A register stores a value v from a domain X and offers r ead and write operations. Initially , a register holds a special value ⊥ 6∈ X . When a client C i in vok es a read operation, the register responds with a value v , denoted r ead i ( X ) → v . When C i in vok es a write operation with value v , denoted write i ( X , v ) , the response of X is an acknowledgment, denoted by O K . The sequential specification requires that each read operation from X return the value written by the most recent preceding write operation, if there is one, and the initial v alue otherwise. W e assume that the values written to ev ery particular register are unique, i.e., no value is written more than once. This can easily be implemented by including the identity of the writer and a sequence number together with the stored v alue. 2 In this paper, we consider single-writer/multi-r eader (SWMR) registers, where for every register , only a designated writer may in vok e the write operation, but an y client may in voke the read operation. Sequential consistency . One of the most important consistency conditions for concurrent access is sequential consistency [7], which preserves the real-time order only for operations by the same client. This is in contrast to linearizability , which must preserve the real-time order for all operations. Definition 1 (Sequential consistency [7]). A history σ is sequentially consistent w .r .t. a functionality F if it can be extended (by appending zero or more response ev ents) to a history σ 0 , and there exists a sequential permutation π of complete ( σ 0 ) such that: 1. For ev ery client C i , the sequence π | C i preserves the real-time order of σ ; and 2. The operations of π satisfy the sequential specification of F . Intuiti vely , sequential consistency requires that e very operation tak es ef fect at some point and occurs some where in the permutation π . This guarantees that ev ery write operation is eventually seen by all clients. In other words, if an operation writes v to a register X , there cannot be an infinite number of subsequent read operations from register X that return a value written to X prior to v . W ait-freedom. A shared functionality needs to ensure li veness. A common requirement is that clients are able to make progress independently of the actions or failures of other clients. A notion that formally captures this idea is wait-fr eedom [5]. Definition 2 (W ait-free history). A history σ is wait-free if ev ery operation by a correct client in σ is complete. F ork sequential consistency . The notion of fork sequential consistency [11] requires, informally , that when an operation is observed directly or indirectly by multiple clients, then the history of ev ents occurring before the operation is the same at these clients. For instance, when a client reads a value written by another client, the reader is assured to be consistent with the writer up to its write operation. Definition 3 (F ork sequential consistency). A history σ is fork-sequentially-consistent w .r .t. a func- tionality F if it can be extended (by appending zero or more response ev ents) to a history σ 0 , such that for each client C i there exists a subsequence σ i of complete ( σ 0 ) and a sequential permutation π i of σ i such that: 1. All complete operations in σ | C i are contained in σ i ; 2. For ev ery client C j , the sequence π i | C j preserves the real-time order of σ ; 3. The operations of π i satisfy the sequential specification of F ; and 4. ( No-join ) For ev ery o ∈ π i ∩ π j , it holds that π o i = π o j . A permutation π i satisfying these properties is called a view of C i . Note that a view π i of C i contains at least all those operations that either occur at C i or are apparent from C i ’ s interaction with F . A fork-sequentially-consistent history in which some permutation π of complete ( σ 0 ) is a possible vie w of all clients is sequentially consistent. W e are now ready to define a fork-sequentially-consistent storage service. It should guarantee se- quential consistency and wait-freedom when the server is correct, and fork sequential consistency oth- erwise. 3 Definition 4 (W ait-free fork-sequentially-consistent Byzantine emulation). A protocol P is a wait- free fork-sequentially-consistent Byzantine emulation of a functionality F on a Byzantine server S if P satisfies the follo wing conditions: 1. If S is correct, the history of e very admissible execution of P is sequentially consistent w .r .t. F and wait-free; and 2. The history of every admissible e xecution of P is fork sequentially consistent w .r .t. F . W e show next that wait-free fork-sequentially-consistent Byzantine emulations of SWMR registers are impossible. 3 Impossibility of W ait-Fr eedom with F ork Sequential Consistency Theorem 1. Ther e is no wait-free fork-sequentially-consistent Byzantine emulation of n ≥ 2 SWMR r e gisters on a Byzantine server S . Pr oof. T ow ards a contradiction assume that there exists such a protocol P . Then in any admissible ex ecution of P with a correct server , ev ery operation of a correct client completes. W e next construct three executions α , β , and γ of P , shown in Figures 1 – 3. All three executions are admissible, since clients issue operations sequentially , and every message sent between two correct parties is ev entually deli vered. There are two clients C 1 and C 2 , which are always correct, and access two SWMR registers X 1 and X 2 . Protocol P describes the asynchronous interaction of the clients with S ; this interaction is depicted in the figures only when necessary . Execution α . In execution α , the server is correct. The execution is shown in Figure 1 and begins with four operations by C 2 : first C 2 ex ecutes a write operation with v alue v 1 to register X 2 , denoted w 1 2 , then an operation reading register X 1 , denoted r 1 2 , then an operation writing v 2 to X 2 , denoted w 2 2 , and finally again a read operation of X 1 , denoted r 2 2 . Since S and C 2 are correct and P is wait-free with a correct server , all operations of C 2 e ventually complete. Figure 1: Execution α , where S is correct. Execution α continues as follows. C 1 starts to execute a single write operation with value u to X 1 , denoted w 1 . Ev ery time a message is sent from C 1 to S during this operation, and as long as no read operation by C 2 from X 1 returns a v alue different from ⊥ , the follo wing steps are repeated in order , for i = 3 , 4 , . . . : (a) The message from C 1 is delayed by the asynchronous network; (b) C 2 ex ecutes an operation writing v i to X 2 , denoted w i 2 ; 4 (c) C 2 ex ecutes an operation reading X 1 , denoted r i 2 ; and (d) the delayed message from C 1 is deli vered to S . Note that w i 2 and r i 2 complete by the assumptions that P is wait-free and that S is correct. For the same reason, operation w 1 e ventually completes. After w 1 completes, and while C 2 does not read any non- ⊥ v alue from X 1 , C 2 continues to execute alternating operations w i 2 and r i 2 , writing v i to X 2 and reading X 1 , respecti vely . This continues until some read returns a non- ⊥ v alue. Because S is correct, e ventually some read of X 1 is guaranteed to return u 6 = ⊥ by sequential consistency of the ex ecution. W e denote the first such read by r z 2 . This is the last operation of C 2 in α . If messages are sent from C 1 to S after the completion of r z 2 , they are not delayed. Note that the prefix of α up to the completion of r 3 2 is indistinguishable to C 2 from an execution in which no client writes to X 1 , and therefore r 1 2 , r 2 2 , and r 3 2 return the initial v alue ⊥ . Hence, z ≥ 4 . W e denote the point of inv ocation of w z − 1 2 in α by t 0 . It is marked by a dotted line. Ex ecutions β and γ constructed below are identical to α before t 0 , but dif fer from α starting at t 0 . Figure 2: Execution β , where S is correct. Execution β . W e next define execution β , shown in Figure 2, in which the server is also correct. Execution β is identical to α up to the end of r z − 2 2 (before t 0 ), but then C 2 halts. In other words, the last two write-read pairs of C 2 in α are missing in β . Operation w 1 is inv oked in β like in α and begins after the completion of r 2 2 (notice that r 2 2 is in β since z ≥ 4 ). Because the protocol is wait-free with the correct server , operation w 1 completes. Afterwards, C 1 repeatedly reads X 2 until v z − 2 is returned. Because the execution is sequentially consistent with the correct server , a read of X 2 e ventually returns v z − 2 . W e denote the i -th read operation of C 1 by r i 1 and the read operation that returns v z − 2 by r l 1 . Execution γ . The third ex ecution γ is sho wn in Figure 3; here, the server is faulty . Ex ecution γ proceeds just like the common prefix of α and β before t 0 , and client C 1 in vok es w 1 in the same way as in α and in β . From t 0 onward, the server simulates β to C 1 . This is easy because S simply hides from C 1 all operations of C 2 starting with w z − 1 2 . The server also simulates α to C 2 . W e next explain ho w this is done. Notice that in α , the server receiv es at most one message from C 1 between t 0 and the completion of r z 2 , and this message is sent before t 0 by construction of α . If such a message exists in α , then in γ , which is identical to α before t 0 , the same message is sent by C 1 . Therefore, the server has all information needed to simulate α to C 2 and r z 2 returns u . Thus, γ is indistinguishable from α to C 2 and indistinguishable from β to C 1 . Howe ver , we next sho w that γ is not fork-sequentially-consistent. Consider the sequential permutation π 2 required by the definition of fork sequential consistency , i.e., the vie w of C 2 . As the real-time order of C 2 ’ s operations 5 Figure 3: Execution γ , where S is faulty and simulates α to C 2 and β to C 1 . and the sequential specification of the registers must be preserved in π 2 , and since r 1 2 , ..., r z − 1 2 return ⊥ but r z 2 returns u , we conclude that w 1 must appear in π 2 and is located after r z − 1 2 but before r z 2 . Because w 1 is one of C 1 ’ s operations, it also appears in π 1 . By the no-join property , the sequence of operations preceding w 1 in π 2 must be the same as the sequence preceding w 1 in π 1 . In particular , w z − 1 2 and w z − 2 2 appear in π 1 before w 1 , and w z − 2 2 precedes w z − 1 2 . Since the real-time order of C 1 ’ s operations must be preserved in π 1 , operation w 1 and, hence, also w z − 1 2 , appears in π 1 before r l 1 . But since w z − 1 2 writes v z − 1 to X 2 and r l 1 reads v z − 2 from X 2 , this violates the sequential specification of X 2 ( v z − 2 is written only by w z − 2 2 ). This contradicts the assumption that P guarantees fork sequential consistency in all ex ecutions. 4 Conclusions When clients store their data on an untrusted server , strong guarantees should be provided whene ver the server is correct, and forking conditions when the server is faulty . Since it was discovered that fork-linearizability does not allow for protocols that are wait-free in all ex ecutions where the server is correct [2], the weaker condition of fork sequential consistenc y was expected to be a promising direction to remedy this shortcoming [2, 11]. In this paper we proved that this is not the case, and in fact, fork sequential consistency suf fers from the same limitation. Refer ences [1] H. Attiya and J. L. W elch. Sequential consistency versus linearizability . A CM T ransactions on Computer Systems , 12(2):91–122, 1994. [2] C. Cachin, A. Shelat, and A. Shraer . Efficient fork-linearizable access to untrusted shared memory . In Pr oc. 26st A CM Symposium on Principles of Distributed Computing (PODC) , pages 129–138, 2007. [3] Collabnet, Inc. Subversion project. http://subversion.tigris.org/ , Last accessed Apr . 2008. [4] Google, Inc. Google Docs. http://docs.google.com/ , Last accessed Apr . 2008. [5] M. Herlihy . W ait-free synchronization. A CM T ransactions on Pr ogramming Languages and Systems , 11(1):124–149, Jan. 1991. [6] M. P . Herlihy and J. M. Wing. Linearizability: A correctness condition for concurrent objects. A CM T ransactions on Pr ogramming Langua ges and Systems , 12(3):463–492, July 1990. [7] L. Lamport. How to make a multiprocessor computer that correctly executes multiprocess programs. IEEE T ransactions on Computers , 28(9):690–691, 1979. 6 [8] J. Li, M. Krohn, D. Mazi ` eres, and D. Shasha. Secure untrusted data repository (SUNDR). In Pr oc. 6th Symp. on Operating Systems Design and Implementation (OSDI) , pages 121–136, 2004. [9] N. A. L ynch. Distributed Algorithms . Morgan Kaufmann, San Francisco, 1996. [10] D. Mazi ` eres and D. Shasha. Building secure file systems out of Byzantine storage. In Pr oc. 21st ACM Symposium on Principles of Distributed Computing (PODC) , pages 108–117, 2002. [11] A. Oprea and M. K. Reiter . On consistency of encrypted files. In Pr oc. 20th Intl. Symp. on Distributed Computing (DISC) , volume 4167 of Lectur e Notes in Computer Science , pages 254–268, 2006. [12] M. Pease, R. Shostak, and L. Lamport. Reaching agreement in the presence of faults. Journal of the A CM , 27(2):228–234, Apr . 1980. [13] W ikipedia. List of file systems, distributed file systems section. http://en.wikipedia.org/wiki/ List_of_file_systems#Distributed_file_systems , Last accessed Apr . 2008. [14] J. Y ang, H. W ang, N. GU, Y . Liu, C. W ang, and Q. Zhang. Lock-free consistency control for web 2.0 applications. In Pr oc. 17th Intl. Confer ence on W orld W ide W eb (WWW) , 2008. 7