Is Privacy Controllable?

Is Priv acy Con trollable? Y efim Sh ulma n (  ) [ 0000 − 0002 − 3163 − 9726] and Joachim Meyer [0000 − 0002 − 1801 − 9987] T el Aviv Universit y , T el Aviv 69978 01, Israel { efimshul man@mail.,jmeyer@ } ta u.ac.il Abstract. One of the ma jor views of priv acy associates priv acy with the contro l over information. This give s rise to the question how control - lable priv acy actually is. In this p ap er, w e adapt certain formal methods of con trol theory and in vestigate th e i mp lications of a con trol theoretic analysis of priv acy . W e look at how con trol and feedbac k mec hanisms hav e b een studied in the priv acy litera tu re. Relying on the control the- oretic framew ork, w e devel op a simplistic conceptual contro l model of priv acy , form ulate priv acy control lability issues and suggest directions for p ossible research. Keywords: Priv acy · F eedback · In formation Disclosure · Human Con- trol · Closed-Loop Con trol · F eedbac k Control 1 In tro duction Casually used in collo quial conversations, the term “pr iv acy” in a ll its com- plexity and pro minence app ea rs in philosophical, legal, p olitical, scientific a nd techn o logical dis cussions. Although there exist numerous definitions, So lov e [46] states in his A T axonomy of Privac y tha t “Priv acy is a concept in disarray”, which “suffers fro m an embarrassment o f meanings” (p. 4 77). Incidentally , the situation has hardly improv ed ever since. While sc holar s struggle with priv acy definitions, the general public (t he users) struggle with their online priv acy settings, as has b een abundantly demonstrated in the security and priv acy literature. The seeming futilit y of information con trol and the lack of functional transpar ency lead p eople to feel helpless. 1 In spite of the diversit y of approa ches, many discuss ions of priv acy tie it to some form of control (control of acces s to informa tion, use of infor mation, distribution of information, etc.). W e take these terms litera lly . If mainstream priv acy research embraces the understanding o f priv acy as con- trol, is there a w ay to analyse the con trolla bility of priv acy? Can w e b orrow from formal m etho ds of con tr ol theory to broaden our understanding of priv acy issues, at least when it is appropr iate to define priv acy as control ov er information? This pap er is o rganise d in the following way . Section 2 of this pape r provides the scop e of the pro blem. Here we discuss the extent o f priv acy deter mined by control and the reach of control theory . 1 See [50] for a case of American consumers b eing resigned to giving up their data in exchange for commercial offers, rather than engaging in cost-b en efit analyses. 2 Y. Shulman and J. Meye r In Section 3 of this pap er we apply a control theoretic framework for a con- ceptual analys is of priv acy as control ov er info rmation. Section 4 presents a disc ussion of the applica bility , limitations and r elev ance of our a nalysis to contempora ry res earch in the pr iv acy liter ature. This paper is a first attempt to analyze priv ac y within the framework of control theo ry . W e map co nt r ol theory onto priv acy , manifested as co nt r ol ov er per sonal information fro m a user’s p ersp ective. Our analysis lo oks a t priv acy on a micro lev el, dealing with the topic in the meaning with whic h it is used in so cia l and computer science discussions (as opp osed to legal, p olitical and philosophical in terpr etations). Star ting fro m Section 3, we use the ter m “ priv acy” int er changeably and as a shorthand for “p ersona l informa tion” a nd its d is closure. Our analysis can serve a s a conceptual framework for discussio ns of priv acy and its implications in different contexts. 2 Priv acy as Con tr ol o v er Information, Con t rol as a Theory Priv acy is a permea ting concept, whic h has no g enerally accepted definition throughout all disciplines. Priv acy definitions 2 are often formulated thro ugh descriptions of the features and pr op erties of priv acy and ev en by writing off constructs which are not priv acy [45]. Incidentally , a bibliometric analysis of computer and infor mation ethics literature has revealed priv acy as one of three ma jor concepts in that field [1 9]. It must b e no ted, ho wever, that the authors make an unsubstantiated claim ab out differenc es b etw een the American a nd the Europ ean approaches to priv acy , ba sed o n their clustering results, wher e “ data protection” fell into the ethics, r ather than the pr iv acy clus ter. 3 In ontological attempts to determine what priv ac y is, scholars often arrive at the same conclusion: general priv acy is contextual. It may be internalised through different conceptualis ations by different individua ls [4 5]. Additional p eculia rities of the concept o f priv acy come to lig ht , when one is reminded that priv acy and the under lying notions may be r elative. F or example, they may not ha ve simult a neously direct and co rresp onding trans lations into other languag es. Smith et al. ([4 5], p. 99 6) write: “Priv acy corres p onds to the desire o f a pers on to control the disclosur e of p ers onal infor mation [...]” while “[...] confidentialit y corresp o nds to the controlled release of pers onal infor mation t o an information custo dian under an a greement that limits the extent and co nditions under which that information may b e used o r released further ”. Nev er theless, the term “ priv acy p o licy” is c onv entionally used in Englis h. Simult a neously , in any 2 In this pap er, w e are not concerned with formal defin itions of priv acy used in cryptog- raphy and priv acy-en hancing technologi es (i.e., differential p riv acy [15], l -diversit y and ( n,t )-closeness [29], etc.). 3 The observed effect could b e an artefact of th eir literature sample (which was not focused on- and, t hus, migh t not be representativ e of priv acy researc h), and (or) sam- pling metho d (pic k in g selected journals in comput er and information ethics without attending to the geographical an d aut horship scop e of those journals). Is Priv acy Con trollable? 3 softw are a pplication or website in Rus sian the s ame do cument is r eferred to as “p olicy of confidentialit y” (literal translation), when it contains specifica tions o f data pro cessing , data pro tection measures and p erso nal infor mation collection. 4 W e can, howev er, r esort to some general conceptions and pheno mena ob- served in the pr iv acy literature. Thus, ma jor approaches to define priv acy in philosophical, legal and scientific writings include: pr iv acy as co nt r ol, pr iv acy as a right, and priv acy as an econo mic go o d. By and la rge, the meaning of pr iv acy is a ttributed arguably to co ntrol over informa tion, r estriction o f a ccess, human dignity , so cia l relations hip and intimacy ([1 2], [34], [45]). Priv acy as cont r ol is a prominent and distinctive appro ach in philoso phical and legal tho ught, and most definitions include features and prop erties, which are as so ciated with the term “ control”. In fact, ma jor theoreticians of priv acy , including W arren and Br andeis [5 2], F r ied [16], and Parent [36] refer to priv acy as some form of control ov er information. Alan W estin defined pr iv acy a s “the cla im of individuals, groups, o r institu- tions to determine for themselves when, how, and to what exten t information ab out them is communicated to others ” ([53], p. 7). Joseph Kupfer a rgues that “ by pro viding control ov er information a bo ut and access to ourselves, pr iv acy ena bles us to define ourselves so cially in ter ms of int imate relationship” ([28], p. 86). Adam Mo or e der ives the following definition: “A right to priv acy is a right to control acces s to and uses of- places, b o dies, and p ersonal informa tion” ([3 4], p. 421). Perhaps the view of priv acy as control ov er information is so widesprea d, be- cause it resonates more easily (enables op eratio nality) with res earch in informa- tion systems, b ehaviora l and cognitive psychology , and ma rketing mana gement. The term “control” has a s pe cific meaning in engineer ing, whe re it is used within the framework o f control theory (see [4], [1 4] and [30] for the theoretical framework and a pplications). Control theor y is the basis for engineer ing mo dels of control of systems and pro cesses, including those that inv olve a human in the control lo op (see [21] and [42] for applicatio ns to human p er formance). Control theory has b een successfully a pplied to the mo delling of manual control ov er a physical system in human factors res earch. Co ncepts o f control theory hav e b een b or row ed b y , and hav e b een pro ductively adjusted to the field of social psyc holog y [32]. Optimal con trol models in economics b elong to a f a mily of optimal control strategies of control theory . The us e of computational mo dels of behavior, including con tro l theo ry , is a dvocated b y psychology scholars in management and organisation sc ience [51 ]. Con trol theory has found its wa y int o life sciences [8], as an alter native form o f Po wers’ p erceptual co ntrol theory (PCT) 5 , whe n the goal o f a dynamic system re sides within the system itself (see 4 In realit y , “priva cy ” is directly translated as “p riva ten ess”, while t he latter cor- respond s to a “degree of inviolabili ty of p riv ate life”, whereas, in fact, “priv acy” corresponds to severa l con trol-, protection- or jurisprudence-related terms in the Russian language (confid ential ity being on e). 5 Originates in [39]. 4 Y. Shulman and J. Meye r [7] for a survey of biologica l, neuro biologica l and ps ychological implementations of negative feedback lo ops). It do es not seem a stretch to assume that pe ople get some for m of feedback o n their b ehavior, including priv a cy-rela ted actions. The inf o rmation p eople receive ab out o utcomes o f their actions ma y alter their b ehavior, which aims to reach some comfort zone, i.e., a certain level o f ph ys ical, men tal o r emotional w ell- being . Of cour se, p eo ple may not always b e able to asso c iate the feedba ck with the a ction (and cause with effect for that matter), a growing concern for priv acy resear chers and data protection profes sionals. With recent developmen ts, it als o bec omes a concern for the genera l public. Control theory is instrumental and pr o ductive when it is applied to pheno m- ena wher e feedback plays some role. In this conceptua l pap er we ask what could be the implications from analyzing priv ac y as con tro l in the f r amework o f con trol theory? Section 3 pr esents our attempt to tackle this question. 3 Con trol Theoretic Analysis of Priv acy Control theory distinguishes b etw een op en-lo op and close d-lo op (fe e db ack) c on- tr ol . In an ope n-lo op control system, some input is fed into the s ystem, and a pro cess runs its course, without fur ther interv entions in the pro cess. In clos ed- lo op (or feedback) c ontrol the output of the pro cess is measured, and s ome information ab o ut the o utput is provided as feedback, s erving to minimize the difference b etw een a desired state and an existing sta te. In the co nt ex t of pr iv acy , o ur s ystem c onsists of: – a per son (the controller) who per forms some actions (e.g., p ermits a n app to a ccess infor mation ab out lo catio n or c ontacts, o r p osts some information on a so cial netw ork); – some pr o cess that runs, dep ending pa rtly on the p er son’s actions; – the cont r olled o utput, whic h is the disclosure of information abo ut the per son or its use; – and the ev aluation of the level of disclosure of p erso nal informatio n 6 . An y part of the pro cess may b e affected by external factors (the environment ) that may introduce noise, or disturbance s. A control theoretic a nalysis of the user a ctions assumes that the output (i.e., the information disclosur e) has so me v a lue that can be compared to a desir ed v alue (e.g., expresse d thro ugh some p ersonally co mfortable level of disclos ure). W e can a ssume that information disc losure ha s s ome b enefits (financial, emo- tional, so cial, etc.) and some p ossible costs. The ov era ll outcome is the s um (or other combination) of the b enefits and the co sts. The exa ct functions by which the benefits and the costs change as more information is revealed, depe nd, of 6 F rom this p oint on, for th e sake of conv enience, we may u se the term “priv acy” as a shorthand for “p ersonal information” and its disclosure. Is Priv acy Con trollable? 5 course, on the p erson, the informa tion, the party rece iving access to the in- formation, and the sp ecific co nt ex t, in which the info rmation is revealed. W e depict a demonstration of the behavior of the con tro lled and output v aria bles in Fig . 1. F o r the sak e of the demo nstration, w e as sume that b enefits incre ase monotonously with diminishing ma rginal returns, and that costs increas e exp o- nent ia lly a s the amount of revealed information increa ses. Utilit y (U) Information disclosure (ID) Benefit (B) Cost (C) Level of Comfort Controlled Variable (P) Optimal comfort 0 U C (ID = P) U B (ID = P) U B (P)+U C (P) Output Fig. 1. Priv acy comfort as utilit y b ased on costs and b enefi t s curves The change in the I nf ormation D iscl osur e ( I D ) leads to desired a nd un- desired consequences for the co ntroller, i.e., b enefits and costs, re sp ectively . Es - timating the difference b etw een c osts and b ene fits for eac h lev el o f ( I D = P ) while exercising control ov er P , a long the abscissa axis, w e seek to maximize the Le v el of C omf or t from disclos ure. The character of co nt r ol, as well a s the optimality criterion will differ, based on the shap es of the benefits and costs functions, as defined by individual and momentary facto rs. W e may assume a more complex scenar io, where the control v aria ble P is m ultidimensio nal: i.e., it ma y con tain multiple t yp es and corresp onding amoun ts of disclosed infor mation ( p [ ty pe,amount ] ∈ P ). The co ntroller wan ts to maximize bo th pleasurable effects and pr iv acy . The optimal co mfort level can b e reached when there is no p ossibility to improv e either of the tw o outcomes, while ma in- taining the same v alue for the other, mapping an optimal O utpu t as a Pareto frontier, as we show in Fig . 2. The Pareto frontier repr esents the Output space, while the a rea under the curve contains sub optimal so lutions that ca n b e improved. The area ab ov e the frontier contains infeasible solutions, due to existing constra int s on the amount of priv acy pr eserved a nd the b enefits g ained for each level of p erso nal informa tion disclosure. 6 Y. Shulman and J. Meye r Rew ard from disclosure Optimal solution Infeasible solution Comfort Frontier Sub- optimal solution Preserved privacy Fig. 2. Priv acy comfort as Pa reto-op t imal frontier F o r the p erson it is desir able to be on the Pareto fro ntier. The p erson has to consider tw o imp ortant questions : (1) Am I on the frontier? If not, what c an I do to get there? (2) Where on the frontier do I prefer to b e? Fig. 3 contains a depiction o f the prop os ed priv acy control model in the form of a blo ck diagram – a widely used wa y to depict dy namic systems in co ntrol theory 7 . – C P A F eedback elements + Reference: desirable comfort + Disturbance on control Control Lagged Control’ Other plausible feedback loops Actions Lagged Actions’ Disturbance on actions Disturbance on system output Output’ Output Ob jective function Lagged Control Error Lagged Actions Lag of control Human controller Actuator: control execution Lag of actions Controlled system/process: privacy Time Delay L A L C Fig. 3. A blo ck diagram representation of the priv acy control model Each b ox in the blo ck diagr am is a separ ate subsystem. A priv acy lev el P is the amoun t of disclosed information, whic h is a cont r olled v ar iable in our 7 Both in d ynamic systems (e.g., [4], [14] and [30]) and human factors (e.g., [21] and [42]) blo ck d iagrams are used for concise dep ictions of systems. Is Priv acy Con trollable? 7 conceptual model. The O ut put 8 of o ur control mo del is the utilit y v ariable, representing the level of comfor t, given the level o f p erso nal disclosure. A human co ntroller C is a p er son, p er forming actions and s eeking to a chiev e some comforta ble level of perso nal information disclosur e. The human controller C adjusts the priv acy level P using an actuator A , which is some for m o f a to ol, system or ser vice through which pr iv acy-related actions are taken (e.g., settings adjustment , information sharing, etc.). Arrows represent signals flowing b etw een the elemen ts o f the s ystem. Arr ows going in to a cer tain element are input signals for this element, and arr ows going out of an elemen t are output signals of this elemen t. An input signal to the whole system is Ref er ence , which is some comfor table level of p ersona l information disclosure that a p erso n desires to achiev e. An output sig nal of the whole system is Outpu t or O utpu t ′ describ ed ab ov e. The circular blo cks are “compa rators ” that sum up inflowing input signa ls, pro ducing output(s). Each tria ngular blo ck repr esents a la g o f output (that is an effect when the output o f the action or pr o cess is not pro p ortional to the input). In termedia te outputs may be non- linear, due to disturbances fro m the en vir onment and the prop erties o f the medium. The or der of the lag is undefined, and the symbol is used for repres entation. A time la g (delay) may also b e present througho ut the system. The most impo rtant time dela y appea rs with the feedbac k loop (sho wn explicitly in Fig. 3). The feedback lo o p includes b oth the infor mation on the re ached comfort v alue ( Outp ut or O utpu t ′ ) and the time delay until the human controller receives the feedback. The time delay is a v arying qua ntit y for ea ch individual at e ach p oint in time, which limits its predictability . W e approa ch the co nceptual pr iv acy co nt r ol mo del through different topics from the sys tems control literature, and we r eveal mult iple controllability issues from the standp oint of the individual, summar ized in T a ble 1 . These issues preclude us from asser ting that h umans and their persona l infor- mation ma ke up controllable systems on their own. Y et, this is no t a reaso n for despair. It only shows that a one-to- one straightforward mapping of a co nt r ol theoretic framework onto p er sonal information disclo sure canno t immediately pro duce b eneficial results. W e discuss p ossible implications and contributions of control theory to pr iv acy in the following Section 4, alongs ide the discussio n on the relev ance and cur rent standings of the notions of “ control” a nd “feedback” in the priv acy literature. In Section 4 we also pr o ceed to discuss the contribution, applicability and limitations of our mo del. W e further inv estigate the existing empirical priv a cy re- search to better understand how our conceptual a nalysis fares with the obser ved reality . 8 Or Output ′ , if any disturbance is in tro duced into t h e system after a certain O utput is ac hieved. 8 Y. Shulman and J. Meye r T able 1. Priv acy controllabilit y issues Issue Description Element F eedback t ime dela y The consequences of actions arrive at an uncertain time, and they ma y b e not attributed to the actions. F eedback loop Physica l feedbac k lag The consequences of actions may arriv e non-linearly and are p rone to alterations within feedbac k elemen ts. F eedback loop Multiple feedbac k lo ops Eac h element of the contro l sy stem (Fig. 3) may hav e its o wn feedbac k loop( s). Model Complexit y Elemen ts of the system may constitute con trol sub- systems with all the corresp onding issues. Model Order of contro l Intermediate signals of controls and actions may h ave different non- linear profiles and may req u ire learning from the human con troller. F orwa rd contro l p ath Multiple physical lags Multiple linear and non-linear relations exist b etw een the elemen ts of the system. Outputs, signals Momen tariness and individual differences The p riv acy control mo del ma y hav e to b e n on- stationary , as p riv acy b ehavior and preferences may v ary o ver time for different ind ividuals. Concept, assumptions Linearit y and time-inv ariance Humans and priv acy are p erhaps non-linear time- v arian t systems: the outpu t is not prop ortional to the input; and at different p oints in time, th e system out - put may differ for th e same system input. Concept, assumptions Is Priv acy Con trollable? 9 4 Discussion In Section 4.1 we present an o verview of, and discussion on how priv acy research has handled the notions o f “ control” and “feedback”, and what b enefits and contributions the c ontrol theoretic analy sis can potentially bring. Section 4.2 describ es p otential resea rch directions, driven by the control theor etic appro ach and cla rifies the scop e and limitations of this pap er. 4.1 Con trol, F e edbac k and Priv acy Researc h The effects and implications of providing users with control (and the feeling of control) ov er their p er sonal informatio n a nd its use hav e b een abundantly stud- ied. Co n tr ol ov er p erso nal information co nstitutes a whole dimensio n of priv acy concerns for use rs ([24], [31]). Interestingly , pe rceived control over information do es no t seem to impact the level of rela ted priv acy concer ns, whenever this control is p erceived to b e low [26]. That result is in line with findings on feeling resigned regarding one ’s priv acy ([50], discussed in Section 2), and that a p er- ceived higher lev el of control may increase the willingness to disclose per sonal information [6]. On a mor e narrow a pproach, it ha s b een s hown that a n incre- men ta l increa se in con tro llability o ver information c ollection may make users more tolerant tow ards tailored o nline advertisemen t [9]. Users may giv e aw ay control ov er p er sonal information as a result of the fra ming o f an online s ervice offer [3]. Additiona lly , ther e a re mu ltiple s tudies in the priv acy decision- making literature that op erationa lize “priv acy control” a nd “ pe rceived priv acy co ntrol” as either dep endent or independent v ariables in their corresp onding models (e.g., [13], [18], [27]; see [43] for a review of more pap ers o n the topic). T echnological implem entations of co ntrol o ver priv acy hav e b een mostly con- cerned with cryptog raphy a nd systems architecture (e.g., [11], [17], [22], [23], [44] and others), or functionality enabling and interface design (e.g., [10], [2 5], [3 3], [40], [47], and many more). 9 How ever, the priv acy control literature so far has used the term “control” mostly a s a mean of adjusting disclosure preferences (e.g ., a djust settings, con- tact support, etc.), a lev el of disclos ure adjustments that ma y b e int r o duced (e.g., ch a nge s ettings and give o r r evok e consent in full or partially , on a level of a server, an applica tion, a lo ca tion, an enterprise, etc.), or as a plausible ad- justmen t that can be made realistically (e.g., s tart disclosing, sto p disclosing, delete information, etc.). Using a control theoretic analy sis, a nd introducing a closed-lo o p control that ca n inform users ab out achieved disclosure outcomes or priv acy states, we can start to use the term “ control” mor e pro ductively . With prop erly asso ciated feedback we may know ab out ac hieved priv acy sta tes and disclosure outcomes. It may enable us to talk ab out “controllability” of pr iv acy states a nd dis closure outcomes. Analysing co ntrollabilit y of priv acy may help us 9 W e invite our readers to ex plore ind ep endently the worl d of p atents on priv acy contro ls. 10 Y. Shulman and J. Meye r answer ques tions a b out whether desir ed states o r o utcomes are reachable, and whether they hav e b een reached. Empirical priv acy resear ch in c omputer science and human-computer inter- action has already given some attention to feedba ck pro cesses and their impact on priv acy b ehavior and p er ceptions (e.g., [41] and [4 9] dealing with feedback design, [37] and [38] lo oking at effects of feedback r ecency). T rying to answer the question of how imp or tant a feedback mechanism ca n be for manag ing p ersonal priv acy , Tsai et al. [4 8] demonstra te that the presence of feedback in a loc ation-shar ing scenario makes p eople feel mor e comfortable with disclosure of p er sonal information and allevia tes the level o f priv acy concerns . Thu s , b oth the aforementioned increase in p erc eived cont r ol and the pres ence of feedback raise the p eople’s information sharing pr op ensity . These findings b ea r risks, alongs ide obvious b enefits, and they s hould b e treated with caution. In a pap er concerned with the state of transpa rency enhancing technologies Murmann and Fischer-H ¨ ubner [35] provide a categorisatio n and ass essment of existing transpar ency enha ncing tec hnolog ies, whic h grea tly rely o n feedback mechanisms. The authors note that without a feedback mechanism, users may be unable to make ratio nal decis ions a b out the use of transpar ency enha ncing techn o logies and ex ercise control ov er them. Hoyle et al. [20] study r elationships b etw een conten t publishers a nd co nten t users. Their findings le ad to the conc lusion that feedback mechanism may b e a useful instrument for balancing p erso nal information disclosur e and exp osur e of the publishers’ conten t. Bargh et al. [5] explore relatio nships b etw een data controllers and data pro- cessors . The authors define “feedback” as an y bac kwards-directed data flo w from data proces sors to data controllers that facilitates for ward-directed data flow. Their conceptua l pa p er is c oncerned with the public p olicy disc ussion on pro ce- dural feedback b etw een different agents dealing with p erso nal data. Discussing nudges in priv a cy and security decis ion-making perfor med with the use of information, Acquisti et al. [2] distinguish b etw een education and feed- back, where education is r esp onsible for affecting future decision-ma king, while feedback is ca pable of altering b ehavior at the curr ent momen t or over time. In terms of control theory , educatio n corresp onds to op en-lo op system dynamics, and feedback naturally rela tes to c lose-lo o pe d sys tems. It m ust b e no ted, how- ever, that the authors use some colloquia l un der standing of the term “feedback”, resulting in a debatable claim that “feedback ca n also inform ab out expected and actual outcomes b efore or immediately after making a decis ion” ([2], p. 44:1 3). A pro ces s that informs ab out “ exp ected outco mes” befo re a n action perha ps constitutes a separate notion (e.g., predictive mo delling, feed-forward control, predictive infer ence, hypothesising , etc. – depending on the context), which is different from what is understo o d by feedback in control theo ry . As we show, the term “feedback” in the pr iv acy literature is often only loosely defined. If w e wan t to pro ceed with analys es in the control theor etic framework, then we should a lign our understanding o f the term “feedback” with the co ntrol theoretic definitions. O ne can use the following definition as an anch o r p oint: Is Priv acy Con trollable? 11 F e e db ack is “ the mo dification, adjustment, or control of a pr o cess o r system (as a so cia l situation or a biologica l mechanism) b y a res ult or effect of the pro ce ss, esp. b y a differ ence b etw een a desired and a n actua l res ult; informatio n ab out the result of a pro cess , exp er iment, e tc.; a resp onse” [1]. Application of the control theoretic framework ma y no t o nly imply the stricter definition of feedback. It is not any information related to priv acy choices that is provided to the decision- maker. Control theoretic feedback r eturns information on achieved lev els of outcomes, which decision-makers can compar e to their own goal levels. This feedback hardly app ea rs in a simple o bvious w ay in rea lity . As was mentioned befor e, the feedback mec hanis m can p erhaps b e imple- men ted in technology . This technology , if it is built with co nt r ol theo retic con- siderations, will differ from existing priv ac y-enhancing technologies and basic recommender systems. Existing systems provide recommendations d er ived from: – profiles of us ers and ass o ciations b etw een pro files and sp ecific users; – the accumulated statistics (historical data) o n priv acy outcomes, whic h allow predictions of desired or unw anted outcomes for sp ecific types o f user s; – the b es t practices and advice from scholars and profes sionals; – the “ raw” information ab out who, how and w hen can, may a nd will access the users’ p ersona l data, if they pro c eed with a given option; – the same “r aw” information a bo ut who, how and when (and p oss ibly for what purp ose) someone actua lly a ccessed s pec ific users’ per sonal data; – and other external data. The desir ed sta te of priv acy , how ever, changes ov er time (a p er son may be- come more informed, more or less conce rned, more or less a lert, etc., as life changes). In o rder to resor t to some comfort levels, a p erso n needs to figure out what priv acy-related actio n to p erform. A p er son would need sufficient un- derstanding of causal a nd temp ora l relationships betw een actions and priv acy- ent a iling consequences, as well as of the c ha racter a nd form of these relatio nships. It is questionable that p eo ple ar e capable and willing to do that. Co nv ersely , ad- justing one’s priv acy to some comfort zo ne ca n b e fac ilitated with the addition of a control-theoretic feedback lo op, providing the following adv antages: 1. P riv acy outcomes of actions may be trace d back to tho se actions in terms of cause and time, through the nature of feedback, accounting for time delays. 2. Desir ed priv acy o utcomes may b e compare d with ac tual pr iv acy outcomes. 3. E ffects of actions on priv acy may b e asso ciated with these actions, even when the form of relatio nships b etw een the actions and their effects is no t prop ortiona l (more co mplex than one-to-o ne mapping, e.g ., “ if-then” rules). This is by acco un ting for physical lag and multiple elements a nd lo o ps. W e face several issues, w hen we a ttempt to mo del the priv acy feedback lo op with control theory: – Is there a wa y for the user o r decis ion-maker to know and define their desir- able o utcomes and states of priv acy? 12 Y. Shulman and J. Meye r – Is there a w ay for the user or decision-maker to asso ciate feedbac k ab out pri- v acy outcomes and implications with actions th a t ha ve led to these outcomes and implications? – Should user s and decision-makers b e nudged tow ards some optimal pr iv acy configuratio n? What would b e the optimality cr iteria in that case? – Should users and decision-makers b e n udge d tow ards some sp ecific priv acy actions? What would b e the justification for and against c ertain actions? – General controllabilit y issues highlighted in T able 1. T echnological implementation of the feedback lo op may help peo ple make better a djustmen ts of their priv acy behavior. The feedbac k may b e partia lly approximated with qua si-linearity , modelle d with anticipation (e.g., a quickened display), Kalman filter and finite state control with time la g and other concepts from c ontrol theory . Alternatively , we may mo del priv acy as an op en-lo o p system. The impr ov e- men t of per sonal disclosure behavior may be achiev ed through enriching p eople’s prior knowledge. One wa y to implement that is to provide re lev an t priv acy ed- ucation, and training. Thu s , the c ontrol theory framework can b e used to come up with an analysis of priv acy and to infor m the developmen t of priv acy s olutions. 4.2 F uture W ork, Scop e and Limitations The control theor etic approach provides v ario us constructs and ideas to be tested in priv acy-r elated research, in cluding priv acy attitudes and b ehaviors, esp ecially decision-making . This co nceptual a nalysis reveals several p otential r esearch dir ections: – Study of feedbac k elemen ts and their effects on priv acy attitudes and be- havior: time lag (delay) for feedback to flow b etw een outcomes and actions , ph y sical lag b etw een a n action and its effect on priv acy , etc. – Study o f rela tions b etw een different elements in a s ystem, involving a user , the user’s priv acy state, the user’s desirable priv acy , information disclosur e outcomes, ev aluations o f outcomes , external factor s, and feedback lo ops b e- t ween these element s . – Study o f users’ decision- making, when feedback lo ops ar e involv ed. – Mo delling certain elements of the priv a cy dec ision-making pro cess in more detail as separate control subs ystems. Some of these subsystems may be suitable for mor e fo rmal control theoretic representation (e.g., application or server p ermissio n mana gement). – Mo delling individual differences in the control theo retic framework. – F ee dback-con tro l lo op implementation in practice. – And, without a doubt, others . W e emphasize t ha t, even though this paper is d evoted to a conceptual control theoretic analysis of priv acy , it can b e naturally developed tow ards mo delling and analyzing priv acy control in information systems. T echnological implementations Is Priv acy Con trollable? 13 of the priv a cy feedbac k lo op, based on control theoretic principles, ma y facilitate individuals’ co ntrol over p ersonal info rmation and its disclos ure and may raise aw areness o f their c urrent priv acy states. An applied control theoretic ana lysis of priv a cy may b e appropriate and particularly v aluable w hen it co mes to the implementation of priv acy-by-design principles. On the one hand, it may help in the ev aluatio n o f a system’s com- pliance w ith the priv acy-by-design principles via as sessing co ntrollabilit y (and stability) o f use rs’ p erso nal information disclosure . It ma y also consult the de- velopmen t of information systems with priv acy-by-design in mind. O n the other hand, an explicit feedback loop mec hanism is easier to develop for a system, which is adhering to the priv acy-by-design principles. W e a lso note that this pap er is no t an exhaustive analysis of priv acy in terms of c ontrol theory . W e did not extend our pap er with a n a lternative analysis, based on Po wers’ p erceptua l control theor y , for the sa ke of keeping the scop e and the rationale of the paper within reaso n and to av oid theor etical debates a round PCT’s assumptions a nd applicability in psyc holo gy . W e also did not ven ture int o the analysis o f op en-lo o p control of priv acy . How ever, analyses o f priv acy in the litera ture so far hav e a lready treated pr iv acy in a so mewhat similar wa y to an o p en-lo op s ystem. It must also b e noted that control theory is instrumental, when we deal w ith closed-lo o p co nt r ol. The presented conceptual analys is of priv acy as an o b ject of co ntrol do es not map the whole b o dy of control theoretic construc ts onto priv a cy r esearch. W e ha ve omitted multiple domain- and application-s pec ific concepts and to ols. Our attempt has b een to ev aluate, tra nsfer and a djust those control theoretic constructs that seem to b ear b enefits and can b e fit to priv acy-rela ted r esearch. F o r the sake of simplicit y we also omitted more formal or specialized asp ects and items (e.g ., underlying pa rtial differential equa tions, Kalma n filter, feed-forward mo dels, etc.), which still may be useful in further analys es o f the sub ject and in relation to sp ecific problems . 5 Conclusion In this pap er we a pply the theoretical framework of cont r ol theo ry to priv acy , according to one of the ma jor understa ndings of priv acy a s a p erson’s co nt r ol ov er information. W e co nceptualize priv acy control with a human controller at its core, and raise questions ab out the co ntrollabilit y o f such a system. The conceptual model of priv acy that we developed and presented in this pap er allows us to re veal m ultiple controllabilit y is sues of pr iv acy , and we prop os e several directions for future res earch with control theory in mind. W e further discuss the relatio n and relev ance of our prop ose d mo del a nd of the control theoretic analysis to priv acy . W e study the existing bo dy o f empirical priv acy rese arch and find mult iple connections in how the priv acy litera ture used and highlighted the notions of control a nd feedback. W e also present our analysis of applicability of the prop o sed appro ach, as well as the challenges and 14 Y. Shulman and J. Meye r opp ortunities of mo delling p ersonal informa tion disclosure as a dynamic sys tem with op en and closed-lo op control. One particula r question we raise concer ns the plausibility of a feedback con- trol lo op of priv acy . If and when the imp lementation of a f eedba ck control lo op is infeasible, priv acy ma y be analyzed as an open- lo op con tro l system. Our analysis shows tha t priv acy may b e a phenomenon that is inherently difficult to control. Some aids can per haps b e used to make it more controllable, such as indications ab out p os sible priv acy implications of actions, o r reco mmendations o n priv acy optimisation thr ough the developmen t of pr iv acy-related feedback control lo ops. F unding. This resea rch is pa rtially funded b y the EU Horizo n 2020 rese arch and innov ation pro gramme under the Marie Sk lo dowsk a- Curie grant agreement No 6 75730 “ Priv acy and Us” . References 1. feedbac k | feed-back, n .: OED Online. Oxford U niversit y Press (July 2018), http://www .oed.com/view/Ent ry/68965 2. Acquisti, A., Adjerid, I., Balebako, R ., Brandimarte, L., Cranor, L.F., Komanduri, S., Leon, P .G., Sadeh, N ., Schaub, F., Sleep er, M., W ang, Y., Wilson, S.: N udges for priva cy and security: Understanding and assisting users’ choices online. ACM Comput. Surv. 50 (3), 44:1–44:41 (Au g 2017). https://doi .org/10.1145/ 3054926 3. Angulo, J., W¨ astlund, E., H¨ ogb erg, J.: What would it take for yo u to tell your secrets to a cloud? In: Bernsmed, K., Fischer-H ¨ ub ner, S. (ed s.) Secure IT Systems. pp. 129–145 . Springer International Publishing, Cham (2014) 4. Astr¨ om, K.J., Murra y , R .M.: F eedback systems: an introduction for scien tists and engineers. Princeton Universit y Press (2010) 5. Bargh, M.S., Meijer, R., Cho enni, S ., Conradie, P .: Priv acy protection in d ata sharing: T ow ards feedbac k b ased solutions. In : Proceedings of th e 8th Internatio n al Conference on Theory and Practice of Electronic Go vernance. pp. 28–36. ICEGOV ’14, A CM, New Y ork, NY, USA (2014). https://doi.org/ 10.1145/2691195.2691279 6. Brandimarte, L., Acquisti, A., Lo ewenstein, G.: Mispla ced confidences: Priv acy and the control paradox. So cial Psychologic al and Pers onality Science 4 (3), 340– 347 (2013). https://doi.o rg/10.1177/19 48 550612455931 7. Carey , T., Mansell, W., T ai, S .: A biopsychosocial mod el based on n ega- tive feedback and c ontrol. F ron tiers in Human Neuroscience 8 , 94 (2014). https://doi .org/10.3389/ fnhum.2014.00094 8. Carv er, C.S.: Con trol p rocesses, priorit y management, and affective dynamics. Emotion Review 7 (4), 301–307 (2015). https:// d oi.org/10.1177 /1754073915 590616 9. Chanc h ary , F., Chiasson, S.: User p erceptions of sh aring, advertisi n g, and trac k ing. In: Elev enth Symp osium On Usable Priv acy and Secu - rit y (SOU PS 2015). pp. 53–67. USENIX Association, Ottaw a (2015), https://ww w.usenix.org/conf erence/soups 2015/proceedings/presentation/chanchary 10. Colnago , J., Guardia, H.: How to inform priv acy agents on p referred level of u ser contro l? I n : Proceedin gs of the 2016 ACM International Joint Conference on Per- v asive and Ubiquitous Computing: Adjun ct. pp. 1542–154 7. UbiComp ’16, ACM, New Y ork, NY, USA (2016). https://doi.org/ 10.1145/2968219.2968546 Is Priv acy Con trollable? 15 11. Coppersmith, D., Mintzer, F.C., T resser, C.P ., W u, C.W., Y eung, M.M.: F ragile imp ercep t ible digital w atermark with priv acy con trol. vol . 3657, p p. 3657 – 3657 – 6 (1999). https://doi. org/10.1117/1 2.344 705 12. DeCew, J.: Priv acy . In: Zalta, E.N. (ed.) The Stanford Ency clopedia of Philosophy . Metaphysics Research Lab, Stanford Universit y , spring 2018 edn . (2018) 13. Dinev, T., Xu, H., Smith , J.H., Hart, P .: Information priva cy and corre- lates: an empirical attemp t to b ridge and d istinguish priv acy-related con- cepts. Europ ean Journal of Information Systems 22 (3), 295– 316 (2013). https://doi .org/10.1057/ ejis.2012.23 14. Do yle, J.C., F rancis, B.A., T annenbaum, A.R .: F eedb ack control theory . Courier Corporation (2013) 15. Dw ork, C.: Differential priva cy : A surv ey of results. In : A gra wal, M., Du, D., Duan, Z., Li, A . (eds.) Theory and Applications of Mo dels of Computation. pp . 1–19. Springer Berlin Heidelb erg, Berlin, Heidelb erg (2008) 16. F ried, C.: An anatom y of v alues: problems of p ersonal choi ce ( 1970) 17. Gong, Y., W ei, L., Guo, Y., Zhang, C., F ang, Y .: Optimal task recommendation for mobile crowdsourcing with p riv acy control. IEEE Internet of Things Journal 3 (5), 745–756 (Oct 2016). https://doi.org/ 10.1109/JIOT.2015.2512282 18. Griffin, C., R a jtma jer, S., Squicciarini, A.: In v ited pap er: A mo d el of parado x- ical priv acy b eh avior in online users. In: 2016 IEEE 2nd In ternational Confer- ence on Collaboration and Internet Computing (CIC). pp. 206–211 (Nov 2016). https://doi .org/10.1109/ CIC.2016.037 19. Heersmink, R., v an den Hov en, J., va n Eck, N.J., v an den Berg, J.: Bibliometric mapping of comput er and information ethics. Ethics and Information T echnology 13 (3), 241 (Apr 2011). https://doi.org /10.1007/s10 6 76-011-9273-7 20. Ho y le, R ., Das, S., Kapadia, A., Lee, A.J., V aniea, K.: Viewing th e view ers: Pub- lishers’ desires and viewe rs’ priv acy concerns in so cial netw orks. In : Proceedings of the 2017 ACM Conference on Computer Supp orted Co op erative W ork and So- cial Comput ing. pp. 555–566. CSCW ’17, A CM, New Y ork, NY , U SA ( 2017). https://doi .org/10.1145/ 2998181.299828 8 21. Jagaci n sk i, R.J., Flach, J.M.: Control theory for humans: Quantitativ e approaches to mod eling p erformance. CRC Press (2003) 22. Jiang, X., La n day , J.A.: Mod eling priv acy con trol in context- a ware systems. IEEE P erva sive Computing 1 (3), 59–63 (July 2002). https://doi .org/10.1109/ MPR V .2002.1037723 23. Jones, S., O’Neill, E.: F easibility of structural netw ork clustering for group-b ased priv acy con trol in social net works. In: Pro ceedings of the Sixth Symp osium on Usable Priv acy and Security . pp. 9:1–9:13. SOUPS ’10, A CM, New Y ork , N Y, USA (2010). https://doi .org/10.1145/ 18 37110.1837122 24. Kitko wsk a, A., W¨ astlund, E., Meyer, J., Martucci, L.A.: Is It Harmful? Re- examining Priv acy Concerns, pp. 59–75. Sprin ger I nternational Pub lishing (2018) 25. Knijnenburg, B.P ., Kobsa, A.: Increasing sharing tend en cy without reducing sat- isfaction: Finding the b est priv acy-settings user interface for social n etw orks. I n: ICIS (2014) 26. Ko walew ski, S., Ziefle, M., Ziegeldorf, H ., W ehrle, K.: Like us on faceb o ok! analyz- ing user preferences regarding priv acy settings in germany . Pro ced ia Man ufacturing 3 , 815 – 822 (2015). https:/ /doi.org/https://doi .org/10.1016/j.promfg.2015.07.336 , http://www .sciencedirect.co m/science/ar ticle/pii/S2351978915003376 , 6th International Conference on App lied Human F actors and Ergonomics (AHFE 2015) and the Affiliated Conferences, AH FE 2015 16 Y. Shulman and J. Meye r 27. Krasno v a, H., Spiekermann, S., K orolev a, K ., H ildebrand, T.: Online so cial net- w orks: wh y we disclose. Journal of Information T echnology 25 (2), 109–125 (Jun 2010). https://doi.org/1 0.1057/jit.2010.6 , https://doi.org/10 .1057/jit.2010.6 28. Kupfer, J.: Priv acy , autonomy , and self-concept. American Philosophical Quarterly 24 (1), 81–89 (1987), http://www.jst or.org/stable/20014176 29. Li, N ., Li, T., V enk atasubramanian, S.: Closeness: A new priv acy measure for data publishing. IEEE T ransactions on Kno wledge and Data Engineering 22 (7), 943– 956 (July 2010). https://doi. org/10.1109/T KD E.2009.139 30. Luenberger, D.G.: Introdu ction to dyn amic sy stems: theory , mo dels, and applica- tions, vol . 1. Wiley New Y ork (1979) 31. Malhotra, N.K., Kim, S.S., Agarwal, J.: Internet users’ information priva cy con- cerns (iuip c): The construct, the scale, and a causal mo del. In fo. Sys. Research 15 (4), 336–355 (Dec 2004). https://doi.org /10.1287/isre.1040.0032 32. Mansell, W., Marken, R .S.: The origins and futu re of control theory in p sycholo gy . Review of General Psycholog y 19 (4), 425–430 (December 2015) 33. Meh ta, V., Bandara, A.K., Price, B.A., Nuseib eh, B.: Priv acy itch and scratc h: On bo dy priv acy war n ings and controls. In: Proceedings of th e 2016 CHI Conference Extended Abstracts on Human F actors in Computing Sys- tems. pp. 2417–2424. CHI EA ’16, ACM, New Y ork, NY, U S A (2016). https://doi .org/10.1145/ 2851581.289247 5 34. Moore, A.: D efining priv acy . Journal of So cial Philosophy 39 (3), 411–428 (2008) 35. Murmann, P ., Fisc her- Hbner, S.: T o ols for achieving usable ex p ost transparency: A survey . IEEE Access 5 , 22965– 22991 (2017). https://doi .org/10.1109/ ACCESS.2017.2765539 36. P arent, W.A.: Priv acy , moralit y , and the law. Philosoph y & Public Affairs 12 (4), 269–288 (1983), http://www. jstor.org/stable/2265374 37. P atil, S., Hoyle, R ., Schleg el, R., Kapadia, A ., Lee, A.J.: Interrupt now or inform later?: Comparing immediate and dela yed priv acy feed- back. In: Pro ceedin gs of the 33rd Annual ACM Confere n ce on H u - man F actors in Comput in g Systems. pp . 1415–141 8. CH I ’15, ACM, New Y ork, NY, U SA (2015 ). https://doi.org/1 0.1145/2702123.2702165 , http://doi .acm.org/10.1145/ 2702123.2702 165 38. P atil, S., Schlegel, R., Kapadia, A ., Lee, A.J.: Reflection or action?: How feed- back an d control affect lo cation sharing decisions. In : Proceedings of the SIGCHI Conference on Human F actors in Computing Systems. p p. 101–110. CHI ’14, ACM , New Y ork, NY, USA (2014). https://doi.org/1 0.1145/2556288.2557121 , http://doi .acm.org/10.1145/ 2556288.2557 121 39. P ow ers, W.T ., Clark, R.K., McF arland, R.L.: A general feedbac k theory of human b eh a v ior: P art i. Perceptual and Motor Skills 11 (1), 71–8 8 (1960). https://doi .org/10.2466/ p ms.1960.11.1.71 40. Sc h aub, F., Balebako , R., Durity , A.L., Cranor, L.F.: A d esign space for effective priva cy notices. I n: Eleven th Symp osium On U sable Priv acy and Security (SOUPS 2015). pp. 1–17. U SENIX Asso ciation, Otta wa (2015), https://ww w.usenix.org/conf erence/soups 2015/proceedings/presentation/schaub 41. Sc h legel, R., Kapadia, A., Lee, A.J.: Eyeing your exposure: Qu antif y ing and con- trolling information sh aring for improv ed priv acy . In: Proceedings of the Sev - enth Symp osium on Usable Priv acy and Securit y . pp . 14:1–14:14. SOU PS ’11, ACM , New Y ork, NY, USA (2011). https://doi.org/1 0.1145/2078827.2078846 , http://doi .acm.org/10.1145/ 2078827.2078 846 42. Sheridan, T.B., F errell, W.R.: Man-machine systems; Information, con trol, and decision models of human p erformance. The MIT Press (1974) Is Priv acy Con trollable? 17 43. Shulman, Y.: T o wards a Broadening of Priv acy D ecision-Making Mo dels: The Use of Cognitiv e Architectures, pp. 187–204 . Sp ringer International Publishing (2018) 44. Siv araman, V., Gharakheili, H.H., Vishw anath, A., Boreli, R., Mehani, O .: Netw ork-level security and priva cy control for smart-home iot dev ices. In: 2015 IEEE 11 th International Conference on Wireless a n d Mobile Comput- ing, Netw orking and Comm un ications (WiMob). pp. 163–167 (Oct 2015). https://doi .org/10.1109/ WiMOB.2015.7347956 45. Smith, H.J ., Dinev, T., Xu, H.: Infor mation priva cy researc h: An in terdisciplinary review. MIS Q. 35 (4), 989–10 16 (Dec 2011), http://dl. acm.org/citation. cfm?id=22089 40.2208950 46. Solo ve, D.J.: A t ax onomy of priva cy . Universit y of Pennsylv ania Law Rev iew 154 (3), 477–56 0 (Jan 2006) 47. T och, E., Cransha w, J., Hankes-Drielsma , P ., S pringfield, J., Kelley , P .G., Cra- nor, L., Hong, J., Sadeh, N.: Locaccino: A priva cy -centric location sharing app li- cation. In: Pro ceedings of the 12th A CM International Conference Adjun ct Pa- p ers on Ub iquitous Comput in g - Adjunct. pp. 381– 382. UbiComp ’10 Adjunct, ACM , New Y ork, NY, USA (2010). https://doi.org/1 0.1145/1864431.1864446 , http://doi .acm.org/10.1145/ 1864431.1864 446 48. Tsai, J.Y., Kelley , P ., Drielsma, P ., Cranor, L.F., Hong, J., Sadeh, N.: Who’s viewed you?: The impact of feedback in a mob ile location- sharing application. In: Proceedings of the SIGCHI Conference on Hu- man F actors in Comput in g Systems. pp . 2003–201 2. CH I ’09, ACM, New Y ork, NY, U SA (2009 ). https://doi.org/1 0.1145/1518701.1519005 , http://doi .acm.org/10.1145/ 1518701.1519 005 49. Tsai, L., Wijesek era, P ., Reardon, J., Reyes, I., Eg elman, S., W agner, D., Goo d, N., Chen, J.W.: T urtle guard: Helping android users apply con tex t ual priv acy preferences. In: Thirteenth Symp osium on Usable Priv acy and Secu- rit y (S O UPS 2017). pp. 145–162. USENIX Asso ciation, Santa Clara, CA (2017), https://ww w.usenix.org/conf erence/soups 2017/technical- sessions/presentation/tsai 50. T urow , J., Henn essy , M.: The tradeoff fallacy: How marketers are misrepresenting american consumers and op ening them up to ex ploitation. SSRN Electronic Journal (June 2015) 51. V ancouver, J.B., W einh ardt, J.M.: Mo deling th e mind and t h e milieu: Computa- tional modeling for micro-level organizational researchers. Organizational Researc h Method s 15 (4), 602–623 (2012). https://doi.org/1 0.1177/1094428112449655 52. W arren, S .D., Brandeis, L.D.: The right to priv acy . H arva rd la w review pp. 193–220 (1890) 53. W estin, A.: Priv acy and freedom. 1967. Atheneum, New Y ork (1970)