Competitive Online Optimization under Inventory Constraints

Competitive Online Optimi zation under Inventory Constraints Qiulin Lin, Hanling Yi ∗ Information Engineering The Chinese Univers ity of Hong Kong John Pang Computing and Math. Sciences California Institute of T echnolo g y Minghua Chen † Information Engineering The Chinese Univers ity of Hong Kong Adam Wierman Computing and Math. Sciences California Institute of T echnolo g y Michae l Honig Electrical Engineering Northwestern Univers ity Y uanzhang Xiao Hawaii Center for Adv . Comm. Universi ty o f Hawaii at Manoa ABSTRA CT This paper studies online opt imization under inve ntory (budget) constraints. While online optimization is a well-studied topic, ve r- sions with inventory constraints have proven difficult . W e con- sider a formulation of inventory-constrained optimization that is a generalization o f the classic one-way trading problem and has a wide range of applications. W e pre sent a new algorithmic frame- work, CR-Pur suit , and prove that it achieves the minimal comp et- itive ratio among all deterministic algorithms (up to a problem- dependent constant fact or) for inventory-constrained online op- timization. Our algorithm and its analysis not only simplify and unify the state-of-the-art re sults for the s tandard one-way trading problem, but they also establish nove l bounds for g eneralizations including concave rev enue functions. Fo r example , for one-way trading with price elasticity , the CR-Pur suit a lgorithm achiev es a competitive ratio t hat is within a small additive co nstant (i.e ., 1/3) to the low er bound of ln θ + 1, where θ is the ratio between the maximum and minimum base prices. CCS CONCEPTS • The ory of computation → Online algorithms ; Design and analysis of algorithms ; • App lied computing → Decision analysis ; KEYWORDS Inve ntory Constraints; Re venue Maximization; Online Algorit hms; One-way Trading; Price Elasticity A CM R efer ence Format: Qiulin Lin, Hanling Yi, John Pa ng, Minghua Chen, Adam Wierman, Michael Honig, and Yua nzhang Xiao. 201 9. Com petitive Online O ptimization under Inventory Constraints. In Proceedings of A CM Sigmetrics conference. ACM, New Y ork, NY, USA, Article 1, 15 pages. https://doi.o rg/10.475/123_4 ∗ The first two authors cont ribute equally to the work. † Corresponding author . Permission to ma ke digital or hard copies of part or all of this work for personal or classroom u se is granted without fee provided that copies a re not made or distributed for profit or commercial advantage and that copies b ear this notice and the full citation on the first page. Copyrights for third-party components of this work must b e honored. For all other uses, cont act the owner / author(s). ACM Sigm etrics conference , June 2019, Phoenix, AZ USA © 2019 Copyright held by the ow ner/author(s). A CM ISBN 123-4567-24-567/08/06. https://doi.org/10.475/123_4 1 IN TRODUCTION Online optimization is a foundational topic in a variety o f com- munities, fr om machine learning t o control t he ory to operations resear ch. There is a large an d active community studying online optimization in a wide ran ge of settings, b oth lo oking at theoret- ical analys is and real-world applications. The applications of o n- line opt imization are wide ranging, e .g., multi-armed bandits [7, 15, 51 ], n etwork op timization (with packi ng constrain ts) [31, 32], data center capacity management [39, 43, 49], smart grid control [41, 47, 5 7], and bey ond. Further , a dive rse set of alg orithmic frame- works have b een deve lope d for online optimization, from the use of classical p otential functions, e.g., [1, 30], to primal-dual t e ch- niques, e.g., [31 , 53], to approaches based on receding horizon con- trol, e .g., [44, 45]. Additionally , many variations of online optimiza- tion have been studied, e. g., online optimization with switching costs [12, 3 8, 40], online optimization with predictions [ 18, 38, 41], conve x bo dy chasing [5, 10, 28], and more. In this pap er , we fo cus on an important class of online optimiza- tion problems that has prov en challenging: online opti m ization un- der i nventory (budget) constraints ( OOIC) . In these problems a de- cision maker has a fixed amou nt of inve ntory , e.g., airlines selling flight ticke ts or battery owners participating in p ower contingency reserves market, and must make a decision in each of t he T rounds with the goal of optimizing p er-round re venue functions. The chal- lenge is that the decision mak er does not have kno wledge of future re venue functions or when the final round will o ccur , i.e ., the value of T . Furt her , the str ict inventory constraint means that an action now has consequences for futu re rounds. As a result of this en- tanglement, positive results have only b een possible for inv entory constrained online optimization in special cases to this point, e.g. , the one-way trading problem [23]. More formal ly , a decision maker in an OOIC participates in T rounds, without knowing T ahead of t ime. In each round, the deci- sion maker selects an action v t ≥ 0, e .g., an amount to sell, after observing a concave reve nue function д t (· ) . Though the decision maker obser ves the reven ue function each round before choosing an action, it is typically not desirable to cho ose an action to max- imize the rev enue in each round due to the limited inventory ∆ . Specifically , the actions are co nstrained by Í T t = 1 v t ≤ ∆ , and con- sequently an action taken at time t constrains future actions. In particular , if the inven tory is used too early then better rev enue functions may app ear later , when inventory is no longer available. A C M Sigmetrics conference, June 2019, P hoenix, AZ USA Q . Lin, H. Yi, J. Pang, M. Chen, A. Wierman, M. Honig, and Y . Xiao OOIC generalizes many well-known online learning and re v- enue maximization problems. One of t he most prominen t is the one-way trading problem [23], where a trader owns some assets (e .g., dollars) and aims to exchange them into other assets (e .g., yen ) as much as possible, depending o n t he price (e.g. , exchan ge rate). There is a long histor y of work on one-way trading [ 19, 21, 23, 29, 42, 56], as w e describe in Sec. 7, and OOI C includes bo th the classic one-way trading problem and variations with concave re venue functions and price elasticity . Applications. Be yond the one-way trading problem, OOIC also captures a variety of other applications. T hree e xamples that hav e motivated our interest in OOIC are (i) power contingency reserve markets [2, 52], (ii) network spectr um trading [ 13, 48], and (iii) o n- line advertisement. In p ower contingency r eser ve markets, the s ystem o p erator faces a contingency , e.g., shortfall of supply t hat may lead to cascading blackouts, and communicates this need to e ither supplement t he power system using battery or cut down large scale p ower supply . Consider the perspect ive of a battery suppl y owner that is decid- ing when to take part in a contingen cy . A contingency may be solved immediately , or it may instead cause a larger contingency whereby the system operator is willing to pay more at a later t ime epo ch. In preparation to p ar t icipate in these contingen cies, batt er- ies are charged earlier and therefore the margin al cost of participa- tion only manifests as an opp ortunity cost against future participa- tion in the day . These situations highlight the need for t he online properties considered in our w ork: (i) the u nknown endin g time T , (ii) future re ven ue functions are not known, and (iii) a co st less, strict inven tory constraint. Similarly , in sp ectrum trading, the owner o f a spectr um band sells bandwidth to make sure that profit or rev enue is maximized give n the investmen ts that have already be en made t o pro cure the particular bandwidth. This means that any cost with r egards to sales only appears as opportunity cost against future possible sales. Similarly , a p otential buye r who is turned down may seek bandwidth from a different provider , and may never return, o r sit- uations may chan ge b etween time epochs, highlighting the same three properties as before: (i) the unknown ending t ime T , (ii) fu - ture rev enue functions are not known, and (iii) a costless and strict inve ntory constraint. In online advertiseme nt, an advertiser wit h a giv en budget would like to inv est into keywor ds from Internet sear ch engines, e.g ., Google AdW or ds. Potential keywords come in an online fashion and may be unavailable at any t ime. It has also been shown in [22] that re venue can b e modell e d as a concave function with respect to the inve stment. The advertiser needs to decide how to in vest its budget fo r keywords to maximize t he overall rev enue , o nce again highlighting the same three properties listed above. Contributions. In this paper we d evelop a new algorithmic framew ork, cal l ed CR-Pur suit , and apply it to dev elop online al- gorithms for the OOIC pr oblem with the optimal co mpetitive ra- tio (up to a problem-dependent constant factor). Further , we pro ve that CR-Pursuit provides the first positive results for a generaliza- tion of the classic one-way trading problem with concave reven ue functions and price elasticity . In more detail, we summarize our contributions as follows. First, w e intr oduce a ne w algor ithmic framework, CR-Pursuit , in Sec. 5. The framew ork is based on the idea of “pu rsuing” an optimized competitive ratio at all time. The framework is parame- terized by a tigh t upp er bound on t he competitive ratio, which is then “pursued” with the actions in each round. W e apply the frame- work to OOIC and generalizations of the one-way trading problem in this pap er , but the framew ork has the p otential for broad appli- cability beyond t hese settings as well. A l ong the way , we also de- rive sev eral useful results on t he offline optimal solution in Sec. 4, which may be o f independent interest. Second, in Sec. 6, we appl y CR-Pursuit to the OOIC problem to achiev e the optimal comp etitive ratio among all d eterministic al- gorithms ( up to a problem-dependent constan t factor) . T o obtain these b ounds we use two technical ideas t hat are of general inter- est bey o nd OOIC. First, we prov e that it suffices to focus on the single-parametric CR-Pursuit algorithm for achie ving the optimal competitive ratio , thu s significantly reducing the search space of optimal online al gorithms. Seco nd, we identify a “critical” input sequence that highlights an important structural property of the space o f input sequences. By applying C R-Pursuit to this critical sequence, we characterize a lower b ound o n the optimal compet- itive ratio as ln θ + 1 where θ is the ratio between the maximum and minimum base prices to be defined in Sec. 3. Subsequently , for any ot her input, the performance ratio achie ved by CR-Pur suit is upper bou nded by the product of a problem-dependent factor and the lower bound. This structure not only suggests a principled ap- proach to characterizing the optimal competitive ratio, but also im- mediately shows that CR-P ursuit achieves the optimal competitive ratio (up to a problem-dependent facto r) amon g all deterministic algorithms. Third, we apply CR-Pursuit to one-way trading problems in Sec. 7. The novel framework simplifies and unifies the state-of- the-art results of the classic one-way trading p roblem. In particu- lar , the critical input d iscussed above is simply the worst case o ne for classical one-way trading; hence, CR-Pur suit achie ves the op- timal competitive ratio ln θ + 1. Further , we show that CR -Pursuit performs well for generalizations of one-way t rading where no positive results wer e previously known. Spe cifically , for one-way trading with price elasticity and concave rev enue functions, CR- Pursuit achie ves a competitive ratio t hat is within a small additive constant (i.e ., 1 / 3) to t he general lower bound of ln θ + 1. 2 RELA TED W ORK. Online opt imization is a large and rich research area and excellent surveys can b e found i n [3, 25]. W ell-known problems in the online optimization paradigm include the classic secretary pr oblem [20], the ski rental problem [36], the one-way t rading problem [23], and the k -serve r problem [27]. Our results represen t the most general results to date for a situat ion where act ions are subject t o a fix ed inve ntory constraint. The p roblem consider ed here is a generalization of the classical one-way trading problem, which has received co nsiderable atten- tion, e.g., [19, 21, 23, 2 9, 4 2, 56 ]. In the one -way trading problem an online decision maker is sequ entially pres ented with exchan ge rates within a bounded region, and she desires to trade all her as- sets to another . The amount of assets trade d in a single time p erio d Competitive Online Optimization under Inventory Constraints A C M Sigmetrics conference, June 2019, P hoenix, AZ USA is assumed to be small enough to not affect the eventual price. El- Y aniv et. al. [ 23] prop ose a threshold-based online algorithm with competitive ratio O ( ln θ ) . Any remainin g items must b e sol d at the last epo ch as that is reven ue maximizing. On the other hand, o u r analysis allows for leftover inventory (sin ce selling al l assets at the last time step may not b e re venue max imizing solution for the last time step in the presence o f p rice elasticity or concave rev- enue functions) and an unknown st opping time, while retainin g the comp etit ive ratio. V ariants of the one-way trading problem hav e been st u died in the l iterature. Chin et al. [19] and Damaschke et al. [2 1] stud y the o ne-way tr ading problem with u nbounded prices and time- varying price bounds, respectively . Zhang et al. [ 56] study the prob- lem when every two consecutive prices are interrelated. Fujiwara et al. [29] stu dy the problem using average-case competitive anal- ysis under the assumption that the distribution of the maximum exchang e rate is known. Kakade et al. [35] incorp orate market vol- ume information and study another one-way trading model in the stock market, call e d the price-volume trading problem. While t he classical one-way trading problem mostly deals with linear rev- enue functions, we note that in our problem we co nsider ge neral concave reve nue functions, which allows us to capture a b oarder class of interesting settings, e. g., one-way trading wit h price elas- ticity . Bey o nd t he one- way trading problem, OOIC is also highly re- lated to ge neralizations of the secretary problem and prophet in - equalities, e.g., [9, 24, 50]. Strong positive results have been ob- tained for t hese problems; howe ver the analytic setting co nsidered differs dramatically from t he current paper . Sp ecifically , we con- sider a worst case analy sis whereas analysis of the secretary prob- lem and prophet inequalities focus on stochastic instances. Under the stochastic setting, so-called “thresholding” algorithms are ef- fective; howe ver such algorithms have unb ounded competitive ra- tios in the worst case setting, ev en under the simplest assumptions. Prior to this work, the most gen eral results known for online problems with inv entory constraints are for the class of problems termed online optimization with packing constraints, e.g., [ 6, 8, 11, 16, 1 7]. This st ream of work developed an intere sting algorithmic framew ork based on a primal-dual or multiplicative weights u p - date approaches, which centers a round maintaining a dual variable for each constraint, understoo d as a shadow (or pseudo) price for the constraint given the information t hus far . While the inv entor y constraints we consider are packi ng constraints, o ur for mulation is fundamentally different than the fo rmulation considered in t hese papers. In these papers, the constraints come in an o nline fashion; whereas in ou r work, the rev enue functions arrive in an online fashion. Another r elat ed o nline o ptimization problem is the k -search prob- lem, where a player searches for the k highest prices in a seq uence that is revealed to her se quentially . When k → ∞ , t he k -max search problem b ecomes the one-way trading problem [42]. Lorenz et. al. [42] p ropose optimal determin istic and randomized online algorithms for bo th the k -max search and k -min sear ch problem. This is differe nt from the well-known k -server problem, where an online algorithm must control the mov ement of k se rvers in a met- ric space t o minimize the movemen t (or latency involved) in serv- ing future requests. A popu lar algorithmic framework for the k - server problem is the p otential function framework . In contrast t o our CR-Pursuit approach, the p otential function approach r equires a b ound b etween the offline optimal cost and the o nline co st at each time epo ch with respect to the p otential. Finally , it is important to distin guish ou r work from t he litera- ture stu dying regret in online optimizatio n, e.g ., [18, 34]. While re- gret is a natural measure for many online op t imization problems, when inventory co nstraints are present it is no longer appropri- ate to compare against the b est static action, as is done by r egret. Static actions ar e p oo r choices when opt imizing reven ue su bject to inv entory constraints. Instead, competitive ratio is the most ap- propriate measure. Further , note that there is a fundamental algo- rithmic t rade-off between optimizing regr et and competitive ratio, ev en when inventory constraints ar e not present. In particul ar, [4] shows that no algorithm can obtain both sub-linear regr et and con- stant competitive ratio. 3 PROBLEM F ORMULA TION W e stud y an online optimization problem wher e a decision maker sells inven tor y across an interval of discrete time slo ts in order to maximize t he aggre gate rev enue. T he reve nue functions of indi- vidual slots are reve aled seq uentially in an online fashion, and the interval length is unknown to the decision mak er . The initial inve n- tory is give n in advance as a constraint, and it bounds the decision maker’s aggr egate selling quantities across time slots. 1 The key notations used in this pap er ar e summarized in T ab. 1. Through- out this paper , we use [ n ] t o repres ent the set { 1 , 2 , .. ., n } w here n is a p ositive integer . More specifically , at t ime t ∈ [ T ] , up on observing the reven ue function д t (· ) , the decision maker has to make an irrevocable de- cision on an action (quantity) v t . Upon cho osing v t the d ecision maker receives a rev enue of д t ( v t ) . The overall objective is t o max- imize the agg regate reve nue, while respecting t he inventory con- straint Í t ∈[ T ] v t ≤ ∆ . 2 W e assume that д t (· ) , ∀ t ∈ [ T ] , satisfy the following conditions: • д t ( v ) is concave , increasing, and differ entiable over [ 0 , ∆ ] ; • д t ( 0 ) = 0 ; • p ( t ) , д ′ t ( 0 ) > 0 and p ( t ) ∈ [ m , M ] . The first condition is a smoot hness condition on the rev enue function and a natural diminish ing return assumption. It also limits our discussion in the more interesting setting where at each time, selling more co uld never decrease rev enue . The second condition implies that selling nothing yields no rev enue. The third condition 1 W e emph asize that in contrast to the works on online optimization with packing constraints [6, 8, 11, 16, 17], the uncertainty in our optimization problem is not that the inventory constraint is unknown before ha nd, but rather the revenue functions arrives in an o nline fashion. 2 The assumption that the a ction is chosen after observi ng the function differ s from the classical online convex o ptimization lit erature [34, 37] , but matches the literature on online convex op timization with switching costs [12, 38, 40] and the literature o n competitive algorithm design, i ncluding those on b u y-or-rent decision making prob- lems [36, 43, 57] and metrical task s ystems [14, 26, 41]. It allows a n is olation of the inefficiency resulting from invent ory constraints rather than also including the i nef- ficiency resulting from t he of lack of knowled ge of the function. A C M Sigmetrics conference, June 2019, P hoenix, AZ USA Q . Lin, H. Yi, J. Pang, M. Chen, A. Wierman, M. Honig, and Y . Xiao T able 1: S ummar y of Notations. T The number of t ime slots ∆ The initial inven tory д t ( v ) The rev enue function of time slot t σ [ 1: t ] Input (rev enue function) sequence up to time t , i.e ., { д 1 , д 2 , .. . , д t } p ( t ) Base price at t ime t , i.e., д ′ t ( 0 ) m , M The lower and upp er bounds of p ( t ) , ∀ t ∈ [ T ] θ The ratio of M / m λ The dual variable asso ciated with the inventory constraint in OOIC v t The selling quantity at time t ¯ v t The selling quantity of CR-Pursuit( π ) at t ime t v ∗ t The optimal selling quantity at time t under the of- fline setting ˆ v t A maximizer of д t ( v ) over [ 0 , ∆ ] Φ ∆ ( π ) The worst case (maximal) inve ntory over all p ossi- ble se quences of inputs need ed to maintain a com- petitive ratio π ≥ 1 for CR-Pur suit ( π ) limits the marg inal reven ue at t he origin (named base price here- after) and en sures that it is b eneficial to sell, since t he base price is po sitive . Denote t he family of all p ossible re venue fun ctions at time t as G . W e assume m and M are known beforehand to the decision maker and denote θ = M / m . W e fo r mulate the problem of online opt imization under in ven - tory constraints (OOI C) as follows: OOIC : max T Õ t = 1 д t ( v t ) (1) s.t. T Õ t = 1 v t ≤ ∆ , (2) var . v t ≥ 0 , ∀ t ∈ [ T ] . (3) Without loss of generality , we assume t hat the inventory constraint in (2) is act ive at t he optimal solut io n. W e can in terpret t he inven tory co nstraint (2) in an OO IC in a parallel way to the inv entory constraint in t he one-way trading problem [ 23]. In particul ar , in the one-way t rading problem the trader has to decide in each slot t he selling quantity v t to maximize the to tal reve nue at the stopping time T . In fact, when sett ing the family of functio ns G to be the family of re ven ue functions of the form д t ( v t ) = p ( t ) v t , we can see OOIC cov ers the one-way trading problem as a special case. Additionally , when addressing reve nue functions of t he form д t ( v t ) = v ( t )( p ( t ) − f t ( v t )) where f t is a conve x function repre senting price elasticity , OOIC repr esents a generalized one-way t rading problem with price elasticity . T o study t he performance of an online algorithm for OOIC we use the comp etitive ratio as the metric of interest. 3 Let A be a d e- terministic online algorithm. It is calle d π - comp etitive if π = max σ ∈ Σ η O P T ( σ ) η A ( σ ) , where Σ is the set of all p ossible inputs ( д t (· ) , t ∈ [ T ] ) and η O P T ( σ ) and η A ( σ ) are the rev enues generated by the op timal offline algo- rithm O PT and the o nline algorithm A , respect ively . This value π is the comp etitive rat io (CR) of the algorithm A . 4 INSIGH TS ON THE OFFLINE SOLU TION In this section, w e derive se veral results on the optimal offline solu- tion. They are useful in the desig n and analysis of our algorithmic framew ork CR-Pur suit in Sec. 5. Under the offline setting wher e д t (· ) , ∀ t ∈ [ T ] , are known in ad- vance to the decision maker , OOIC is a conve x problem and can be solved efficiently . Let v ∗ be the op t imal primal solution and λ ∗ be the o ptimal dual variable associated with the inventory constraint in (2). W e note that λ ∗ can be obt ained by the algorithm in Alg. 2 in Appendix A.1, based on a binary search idea. The following propo - sition giv es a se t of optimality co nditions for the optimal p rimal solutions v ∗ and the optimal dual variable λ ∗ . Proposition 1. Under our setting that the inventory constraint is active at the optimal solution, the optimal pr imal and d ual solutions v ∗ and λ ∗ satisfy (i) λ ∗ ≥ 0 and Í T t = 1 v ∗ t = ∆ and (ii) for each t ∈ [ T ] , ( v ∗ t = 0 , if д ′ t ( 0 ) < λ ∗ ; v ∗ t ∈ V t ( λ ∗ ) , { v t | д ′ t ( v t ) = λ ∗ , v t ∈ [ 0 , ∆ ]} , otherwise . (4) Recall that at time t , the marginal re venue evaluated at v t is д ′ t ( v t ) , which is no larger than t he base price p ( t ) = д ′ t ( 0 ) due t o the concavity of д t (· ) . The o ptimal dual variable λ ∗ can be in ter- preted as the marginal cost (shado w price) of the inve ntory . Then Proposition 1 say s t hat, at the optimal solution, the ma rginal rev- enue must equal the marginal cost in the sl o ts with po sitive sell- ing quantities. Moreov er , it is optimal to sell o nly in the slo ts in which the base price is higher than the opt imal marginal cost , i.e ., p ( t ) > λ ∗ . These o b servations are similar to those in the Cournot competition literature, e.g ., [ 46]. Next, we re veal two interesting obser vations on the offline op- timal aggr egate reve nue. Recall the input (rev enue function) se- quence until time t is σ [ 1: t ] = σ [ 1: t − 1 ] ∪ { д t (· ) } = σ [ 1: t − 2 ] ∪ { д t − 1 (· ) } ∪ { д t (· ) } = · · · . Recall that η O P T  σ [ 1: t ]  is the offline optimal aggregate rev enue give n the input σ [ 1: t ] . The following lemma b ounds the increment of the optimal aggregate reve nue as t increases. Lemma 2. Let λ t − 1 and λ t be the optimal dual vari ables associated with th e inventory constraint given the inputs σ [ 1: t − 1 ] and σ [ 1: t ] = 3 Note that ma ny pa pers in the on line optimization lit erature, e.g., [34], focus on regr et instead of competitive r atio, but regret is no t an a ppropriate measur e when inventory constraints are considered since static action s ar e no longer appropriate. Our focus on competitive ratio matches that of the literature on s ecretary problems [9, 50], prophet inequalities [33, 50], onl ine optimization with switching costs [ 12, 38, 40, 41, 43], etc. Competitive Online Optimization under Inventory Constraints A C M Sigmetrics conference, June 2019, P hoenix, AZ USA σ [ 1: t − 1 ] ∪ { д t (· ) } , respectively . L et ˜ v t be the optimal offline so lution in the (last) tim e slot t given the in put σ [ 1: t ] . T he following inequalities hold: η O P T  σ [ 1: t ]  − η O P T  σ [ 1: t − 1 ]  ≥ д t ( ˜ v t ) − λ t ˜ v t , (5) and η O P T  σ [ 1: t ]  − η O P T  σ [ 1: t − 1 ]  ≤ д t ( ˜ v t ) − λ t − 1 ˜ v t ≤ д t ( ˆ v t ) , (6) where ˆ v t is the maximiz er of д t (· ) ov er [ 0 , ∆ ] . Note that λ t − 1 and λ t are the marginal costs of inventory at the optimal solutions to OOIC given the inputs σ [ 1: t − 1 ] and σ [ 1: t ] = σ [ 1: t − 1 ] ∪ { д t (· ) } , re spect ively . The terms λ t ˜ v t and λ t − 1 ˜ v t repre sent the u pper and lower bou nds of the cost of committ ing ˜ v t to obtain the new additional reven ue д t ( ˜ v t ) in time slot t . Thus t he differ- ence between them repre sents a bou nd o n the “profit” obtained in slot t . Intuitively , Lemma 2 sa ys that one can b ound the o ptimal offline rev enue increme nt by these profit bo u nds, as sho wn in (5) and (6). The proof of Lemma 2 is included in t he Appendix A.3; we give the proof idea here. Compared w ith the optimal solution under σ [ 1: t − 1 ] , the optimal solution under σ [ 1: t ] is smaller at τ , ∀ τ ≤ t − 1, in order to commit ˜ v t to д t (· ) , which cause a decrement in reve nue. Furthermore, the per-unit re venue lost is upper bounded by λ t and lower bou nded by λ t − 1 . Combining the two understandings gives the bou nds of the increment on optimal reve nue at each time. The upper b ound in (6) also highlights an in tuitive res ult that the increme nt o f the optimal aggreg ate r ev enue from t − 1 to t is at most д t ( ˆ v t ) , i.e ., the maximum re venue one can obtain in slot t . Our l ast result in this section, as state d in the lemma below , re- veals another subt le y et important property o f the increme nt of the optimal aggregate rev enue . Lemma 3. Let ˜ σ be an input sequence. ¯ σ is another input sequence constructed by interchanging д τ and д τ + 1 in ˜ σ , for any selecte d τ ∈ [ T ] . W e have η O P T  ˜ σ [ 1: τ ]  − η O P T  ˜ σ [ 1: τ − 1 ]  ≥ η O P T  ¯ σ [ 1: τ + 1 ]  − η O P T  ¯ σ [ 1: τ ]  . (7) The left- (r esp. right-) hand-s ide of (7) can be r egarded as the increme nt д τ contributes t o the offline optimal under ˜ σ (resp. ¯ σ ). Inequality (7) mean s that mo ving д τ + 1 ahead of д τ (as under ¯ σ ) will not increase the contribut ion of д τ to the offline op t imal. Lemma 3 basically states that regardles s of the input seq uence thus far , the impact or impro vemen t in the offline optimal that д τ brings at the time it appears in the input sequence has a “diminis hing effect” in time. The proof of Lemma 3 is essentially based o n the bo unds on the increment of offline opt imal at each time in Lemma 2. W e leave the proof in App endix A.4. 5 CR-P URSUIT ALGORI THMIC FRAMEW ORK In this sect ion, we present CR-Pursuit , a new algorithmic fr ame- work for solving OO IC under the online sett ing, where the interval length T is not known beforehad and the reven ue functions д t (· ) , t ∈ [ T ] , are rev ealed in a slot-by-slot fashion. Algorithm 1 CR-Purs uit ( π ) Online algorithm 1: Input: π > 1 , ∆ 2: Output: ¯ v t , t ∈ [ T ] 3: while t is not t he last slot do 4: Obtain η O P T  σ [ 1: t ]  by solving the conve x problem OOIC give n the input until t , i.e., σ [ 1: t ] 5: Obtain a ¯ v t ∈ [ 0 , ∆ ] that satisfies (8) 6: end while CR-Pursuit is parameterized by a comp etitive rat io π , an d chooses actions w it h t he goal of “pur suing” this competitive ratio, i. e., ma in- taining the comp etitive r at io against the offline o ptimal of the pre- viously ob served rev enue functions at all time. W e derive b ounds on the o ptimal co mp etitive ratios and use them t o operat e CR- Pursuit accordingly . In the foll owing, w e first pr esent the C R-Pursuit frame work and show that one can optimize the only parameter of C R-Pursuit to achiev e the b est p ossible competitive ratio, thus significantly re- ducing the search space of opt imal online algorithms. Then, we identify a “critical” input sequence that highlights an imp ortant structural property o f t he space of input sequences. By applyin g CR-Pursuit t o this critical sequence, we characterize a lower b ound on the optimal comp etitive ratio as ln θ + 1, w here we recall t hat θ = M / m is the ratio between the maximum and minimum base prices. T hen, for any other input, the p erformance ratio achiev e d by C R-Pursuit (with the same parameter) is upper bounded by the product of a problem-dependent factor and t he lowe r bo und. This structure not only suggests a principled approach for character- izing the optimal comp etitive ratio, but also immed iately shows that CR-P ursuit (with a parameter being the product of the lower bound and the problem-dependent factor) achiev es the opt imal competitive ratio (up to a p roblem-dependent factor) for solving OOIC among all deterministic algorithms. CR-Pursuit . Recall that σ [ 1: t ] = { д 1 , д 2 , .. . , д t } is the input up to time t and η O P T  σ [ 1: t ]  is the corresponding opt imal o ffl ine r ev- enue. The class of online algorithms that make up the CR-Pur suit framew ork, denoted as CR-Pur suit ( π ) and presented in Alg. 1, can be described as foll ows: Giv en any π ≥ 1, at the current time t , CR-Pursuit ( π ) o utputs a ¯ v t ∈ [ 0 , ∆ ] t hat satisfies д t ( ¯ v t ) = 1 π h η O P T  σ [ 1: t ]  − η O P T  σ [ 1: t − 1 ]  i . (8) W e remark t hat such ¯ v t always exists, be cause (i) д t (· ) is a contin- uous and increa sing function and (ii) t he right-hand-side of (8) is in [ д t ( 0 ) , д t ( ˆ v t )] according to Lemma 2. Essentially , CR-Pursuit ( π ) aims at keeping t he offline-to-online rev en ue ratio to be π > 1 at all time, i.e., t Õ τ = 1 д τ ( ¯ v τ ) = 1 π η O P T  σ [ 1: t ]  , ∀ t ∈ [ T ] . (9) While CR-Pur suit ( π ) can be defined for any π , the solution ob- tained by CR-Pursuit ( π ) may violate the inv entor y constraint in OOIC and be infeasible. T his motivates the following definition. A C M Sigmetrics conference, June 2019, P hoenix, AZ USA Q . Lin, H. Yi, J. Pang, M. Chen, A. Wierman, M. Honig, and Y . Xiao Definition 4. CR-Pursuit ( π ) i s feasible if Φ ∆ ( π ) ≤ ∆ , where Φ ∆ ( π ) , max σ ∈ Σ T Õ t = 1 ¯ v t ( σ ) , ( 10) and ¯ v t ( σ ) is the output of CR-Pursuit ( π ) at time t und er the input σ . If CR-Pursuit ( π ) is fe asible, i.e ., it can maintain t he offline-to- online rev enue ratio to b e π under all possible input sequences without violating the inven tor y co nstraint, t hen by definition it is π -comp etitive. W e presen t a useful observation on Φ ∆ ( π ) . Lemma 5. Φ ∆ ( π ) is strictly de creasing in π over [ 1 , ∞) . Lemma 5 fol lows naturally si nce attempting to preserve a smaller competitive ratio requires selling a larger inventory to match the discounted reve nue obt ained by the offline opt imal algorithm. It also implies that if CR-Pur suit ( π 1 ) is feasible for some π 1 , then any on line algorithm CR-Pursuit ( π ) with π ≥ π 1 is also feasible . Thus an up per bound on the optimal competitive ratio in this case give s a feasible comp etit ive online algorithm. The Optimal Competitive Ratio. W e now p resen t a key re- sult, which says that it suffic es to focu s on CR-Pursuit for achiev- ing the opt imal competitive ratio. Theorem 6. Let π ∗ be the u n ique s olution to the characteristic equa- tion Φ ∆ ( π ) = ∆ . Th en CR-Pursuit ( π ∗ ) is feasible and π ∗ is the opti - mal comp etitive ratio of deterministic online algorithms. Before we proceed to prove T heorem 6, we first pre sent the fol- lowing lemma characterizing a class o f worst case inputs for C R- Pursuit . The results will b e used in the p roof of Theorem 6. Lemma 7. F or any CR-Pursuit ( π ), there e xists an input sequence σ such th at (i ) C R-Pursuit ( π ) sells exactly the Φ ∆ ( π ) amount of inven- tory and (ii) д ′ t ( ¯ v t ( σ )) is non-de creasing in t . Lemma 7 stat es that to compute Φ ∆ ( π ) , it is sufficient to fo- cus on the input sequences that will l ead to a non-incr easing se- quence of marginal rev enue д ′ t ( ¯ v t ) at t he solution obtained by CR- Pursuit ( π ). Intuitively , these se quences will cause the C R-Pursuit ( π ) algorithm to sell large q uantities at lower prices in the early slo ts, without knowing that the marginal re venues at later slo ts are hi gher , which is exploite d by the o ffline optimal solu t ion. As a result, CR- Pursuit ( π ) will need to sell the “worst” amount o f inventory t o keep the reve nue ratio π . The proof of L emma 7 is pro vided in Appendix A.5, based on the subt le ye t impor tant property of the offline opt imal aggre gate re venue in Le mma 3. T he idea of the proof, r oughly speaking , is that if the worst case input sequence is not as stat ed, then we can swap reve nue functions within the sequence to construct a new worst case one that satisfies the conditions. Putting the prece ding lemmas together , we are now ready to prove Theorem 6. Proof of The orem 6. The feasibility of CR-Pursuit ( π ∗ ) is be- cause o f t he definition of π ∗ . What remains to be prov e d is that π ∗ is the optimal comp etitive ratio. Consider an arbitrary deterministic online algorithm different from CR-Pur suit ( π ∗ ) , denoted as A . W e will show that A cannot achiev e an offline-to-online rev enue rat io small er than π ∗ ove r an input sequ ence that we construct. Let ˜ σ [ 1: T ] = { ˜ д 1 , ˜ д 2 , .. ., ˜ д T } b e a w o rst case in put sequence of CR-Pursuit ( π ∗ ) that satisfies the conditions in Lemma 7. Let ¯ v t and v A t be the corresponding sol utions of CR-Pur suit ( π ∗ ) and A at time t , respectively . W e have • Í T t = 1 ¯ v t = Φ ∆ ( π ∗ ) = ∆ ; • ˜ д ′ t ( ¯ v t ) is non-decreasing in t . W e no w constru ct an input sequence over w hich A cannot achieve an offline-to-online re venue ratio smaller than π ∗ , by feeding ˜ д 1 , ˜ д 2 , .. . , ˜ д T to A and stop at any time that we nee d . W e first present ˜ д 1 to A in the first slot. If v A 1 ≤ ¯ v 1 , w e st op and set T = 1 in this constructed seq u ence . In this case , we have ˜ д 1  v A 1  ≤ ˜ д 1 ( ¯ v 1 ) = 1 π ∗ η O P T  ˜ σ [ 1:1 ]  , thus the comp etitive ratio of A is at least π ∗ . Otherwise we have v A 1 > ¯ v 1 and we continue to present ˜ д 2 to A in the second slot . In general, if at time t the t otal selling quantity of A so far is no larger than that of CR-Pursuit ( π ∗ ) , i.e., Í t τ = 1 ¯ v τ , we end the trading period. Otherwise, we continue to the t + 1 slot and p resent A with the reven ue function ˜ д t + 1 (· ) . Let τ be the earliest slot such that at the end of time τ , the total selling quantity o f A is less than that of CR-Pursuit ( π ∗ ) . Such τ exists; otherwise, we will have Í T t = 1 v A t > Í T t = 1 ¯ v t = ∆ , w hich implies that A is not feasible. Given such τ ∈ [ T ] , we have t Õ ξ = 1 v A ξ > t Õ ξ = 1 ¯ v ξ , ∀ t ∈ [ τ − 1 ] , (11) and τ Õ ξ = 1 v A ξ ≤ τ Õ ξ = 1 ¯ v ξ . (12) W e now show that, for the input sequ ence ˜ σ [ 1: τ ] , the aggre gate re venue of A is no larger than that of CR-Pursuit ( π ∗ ) , i.e., τ Õ ξ = 1 ˜ д ξ  v A ξ  − τ Õ ξ = 1 ˜ д ξ  ¯ v ξ  ≤ 0 , (13) which t hen implies that the onlin e algor it hm A is at best π ∗ -competitive. By the concavity of ˜ д t (· ) , we have τ Õ ξ = 1 h ˜ д ξ  v A ξ  − ˜ д ξ  ¯ v ξ  i ≤ τ Õ ξ = 1 ˜ д ′ ξ  ¯ v ξ   v A ξ − ¯ v ξ  = ˜ д ′ τ ( ¯ v τ ) ©  « τ Õ ξ = 1 v A ξ − τ Õ ξ = 1 ¯ v ξ ª ® ¬ − τ − 1 Õ t = 1  ˜ д ′ t + 1 ( ¯ v t + 1 ) − ˜ д ′ t ( ¯ v t )  ©  « t Õ ξ = 1 v A ξ − t Õ ξ = 1 ¯ v ξ ª ® ¬ . By (12) and that ˜ д ′ τ ( ¯ v τ ) ≥ 0 as ˜ д τ (· ) is an increas ing function, the first term in the l ast line of derivation is non-positive. By (11) and that ˜ д ′ t ( ¯ v t ) is non-decreasi ng in t , each term in the summation in the l ast line of derivation is non-n egative . As such, the right-han d- side is non-positive and the ineq uality in (13) hol d s.  Competitive Online Optimization under Inventory Constraints A C M Sigmetrics conference, June 2019, P hoenix, AZ USA 6 COMPETI TIV E ANAL YS IS OF CR-P URSUIT The results in t he previous section highlight a principled approach to construct an optimal online algorithm. Spe cifically , the first step is to mathematically c haract erize Φ ∆ ( π ) . T hen we solve the char- acteristic e quation Φ ∆ ( π ) = ∆ to ob t ain the optimal competit ive ratio π ∗ , and CR- Pursuit ( π ∗ ) is an opt imal o nline algorit hm for solving OOIC. For special cases such as the one-way t rading prob- lem [23] where д t ( v ) = p ( t ) · v , we can o btain the closed-for m expres sion of Φ ∆ ( π ) and comput e the optimal comp etitive ratio (as demonstrated in Sec. 7.1). H owe ver , it is difficult to obtain a closed-form expression for general concave rev enue functions. In- stead, we characterize an upp er bou nd on Φ ∆ ( π ) , based on w hich we can giv e an u pper b ound o n the op timal comp etitive ratio π ∗ and consequently a feasible online algorithm. Before moving to the upper bound though, it is helpful to under- stand a lower bound on the optimal competitive ratio. For this, we can simply re fer t o the literatu re on one-way trading. In particular , it has been shown that the opt imal competitive ratio of the clas- sic o ne-way trading problem is ln θ + 1 [23, 54]. Since OOIC cov ers one-way trading as a special case , the optimal comp etit ive ratio for any on line algorithm solving OOIC is low er b ounded by ln θ + 1. Interestin gly , it is possible to interpr et this bound in the conte x t of the CR-Pur suit framework. In particul ar , in Sec. 7.1. we identify the worst case input in one-way tr ad ing ( defined in Sec. 7.1) as a “critical" input sequence, re flecting an interesting stru cture on the space of input sequences. By applying CR-Pursuit to this se q uence, we charact erize a lower bo und o n t he opt imal comp etitive r atio as ln θ + 1. It tu r ns out that for any ot her inputs, t he performance rat io achiev e d by CR-P ursuit is upper bo u nded by the product of a p roblem- dependent factor and the lower bound ln θ + 1 . This insight leads to the following results. Theorem 8. Recall that G is the set of a ll possible д ( ·) and ˆ v ∈ [ 0 , ∆ ] is the maximizer of д (· ) . Let c = sup д ∈ G д ′ ( 0 ) д ( ˆ v )/ ˆ v , then the optimal competiti ve r atio π ∗ satisfies ln θ + 1 ≤ π ∗ ≤ c ( ln θ + 1 ) . Theorem 8 characterizes an upper bound on the optimal co m- petitive rat io in the case for general rev enue functions д t , and also implies that CR-Pursuit ( c ( ln θ + 1 )) is feasible and its competitive ratio is c ( ln θ + 1 ) . Note that c is a constant that dep ends on the gra- dient properties (in particular the base price) and t he maximizers of the rev enue functions 4 . For many inter esting problems, t his c is bounde d and small. For example, for the one-way trading problem where the reve nue functions are linear , i.e. , д t ( v ) = p ( t ) v , ∀ t ∈ [ T ] , we have c = 1. As another example, for the one-way trading with linear price elasticity w here the re venue functions are qua- dratic, i.e., д t ( v ) = ( p ( t ) − α t v ) v , ∀ t ∈ [ T ] , we have c = 2. 4 While c is a c onstant when the family of revenue functions are fixed, it is indeed true that c could presumably be driven to b e infinitely large, e.g., with revenue functions that a r e concave a nd increasing. This parameter c ca n be seen in an economical sense as a comparison between the base price and the a verage price at the maximizer of the function. Since the former is already bounded in [ m , M ] , we look a t the ca se when the latter i s small. These situations are hard to derive a ny i nteresting online optimization as the functions require too much c ommi tment even in bad time epochs, and ha v e low average pric es. This results in low committed avera ge prices while the offline opt imal may eventually not have to partici pate in these time epochs. T o prove this theorem, we use a se quence of lemmas elab orated as foll ows. W e b egin with Lemma 9, which gives an upper bound on the tot al selling quantity by CR-Pur suit ( π ) in each time slot to maintain the offl ine-to-online re venue ratio. Recall that the output of the algorithm CR-Purs uit ( π ) at slo t t , д t ( ¯ v t ) is given in (8), and p ( t ) = д ′ t ( 0 ) is the base price at slot t . Lemma 9. For any i nput sequence σ , we have ¯ v t ≤ c д t ( ¯ v t ) p ( t ) , ∀ t ∈ [ T ] . The proof o f Lemma 9 is included in Ap p endix A.6, by lever- aging the definition of c and that д t ( v ) is an increasi ng concave function. Next, we prese nt an inter esting res ult that bounds t he contribu- tion to the online reve nue in all the slots whose base prices is n o higher than any spe cific threshold. Lemma 10. For any i n put seq uence σ ∈ Σ , for any threshold price p ∈ [ m , M ] , we have Õ { t : p ( t ) ≤ p } д t ( ¯ v t ) ≤ 1 π p · ∆ . Lemma 1 0 is intuitive in that the left-hand-side is the online re venue o b tained by CR-Pursuit ( π ) in the slots whose base prices is not higher than p . The right-hand-side is simply the maximum re venue achieva ble by CR-Pursuit ( π ) in these slots acco rding to its design. In the p ro of, we first observe that if p ( t ) < p , ∀ t ∈ [ T ] , the result is immediate. A s for general cases, based o n Le mma 3, w e can construct ne w input sequences by movin g forw ard the slots with p ( t ) ≤ p in σ , while increasin g t he online reven ue in the slots that we are inter ested in. At last, we obtain an input sequence with larger online rev enue in these slots, which are now all in the beginning of the input se quence. The total online rev enue in them is b ounded by p · ∆ / π . Lemma 10 allows to prove a key step u sed in the proof of Theo rem 8 below . Lemma 11. For any input sequence σ , we have T Õ t = 1 д t ( ¯ v t ) p ( t ) ≤ ∆ π ( ln θ + 1 ) . (14) The idea to prov e Lemma 11 is t o construct an optimization problem, whose optimal objective value b ounds the left-hand-side in (14), subject to the constraint fr o m Lemma 10. Then we sho w the optimal objective value can be further upper b ounded by the right-hand-side in (14). W e are now ready to prove Theorem 8. Proof of Theorem 8. It is clear that CR-Purs uit is at best ( ln θ + 1 ) - competitive, as it covers the one-way trading problem as a special case, which has an optimal comp etitive ratio of ln θ + 1. T o establish the upper bou nd, by Lemmas 9 and 11, we observe Φ ∆ ( π ) = max σ ∈ Σ T Õ t = 1 ¯ v t ≤ T Õ t = 1 c д t ( ¯ v ( t ) p ( t ) ≤ c ∆ π ( ln θ + 1 ) . By solving c ∆ π ( ln θ + 1 ) = ∆ , we get that ¯ π = c ( ln θ + 1 ) and Φ ∆ ( ¯ π ) ≤ ∆ . Then according to t he defini tions, CR-Purs uit ( ¯ π ) is A C M Sigmetrics conference, June 2019, P hoenix, AZ USA Q . Lin, H. Yi, J. Pang, M. Chen, A. Wierman, M. Honig, and Y . Xiao feasible and is ¯ π -competitive. Hence , ¯ π is an upp er bou nd for the optimal comp etit ive ratio π ∗ .  Theorem 8 implies that C R-Pursuit achi ev es the optimal com- petitive ratio (up to a problem-dependen t factor c ) among all d e- terministic online algorit hms. 7 APPLICA TION TO ONE- W A Y TRADING In this section, we apply CR-Pur suit to the classic one-way trading problem [ 23] and its generalizations, illustrating that the frame- work can b oth match state-of-the-art results for the classic setting and pro vide ne w results for generalizations that hav e previously resisted analysis. In particular , using the CR-Pursuit framework, we o btain an onlin e algorithm matching the optimal comp etitive ratio ( ln θ + 1 ) for the cl assic one-way t rading problem in Proposi- tion 13 and a near-optimal ( ln θ + 4 / 3 ) r esult for the case with l inear price elasticity in T heorem 16. F urthermore, the algorithmic frame- work also extends to any convex price elasticity , and yield online algorithms with order-optimal comp etit ive ratio in these cases. This se ction also provides an illustration o f how the framework can be applied to spe cific problem do mains to obtain tighter com- petitive rat io upp er b ounds that the generic ones under general settings. In particular , fo r one-way trading with linear price elas- ticity , the upp er bou nd derived from Sec. 6 is 2 ( ln θ + 1 ) while the bound obt ained in this sectio n is ln θ + 4 / 3. In Sec. 7.1, we obt ain a close-form expre ssion of Φ ∆ ( π ) and com- pute the optimal π ∗ in this special case . In Sec. 7.2, we sho w the ease of generalizing the o ne-way trading problem, to cases where price formation include price elasticity , an aspect that has been left out and desired in the one-way trading community . 7.1 Classic One-way Trading In the classic one-way trading problem, a t rader own s so me assets (e .g., one dollar) at t he b eginnin g and aims to exchan ge it into an- other assets (e .g., yen) as much as possible, depending on the price (e .g., exchange rate). Thus, the o ne-way trading problem is a spe- cial case of the OOIC p roblem with д t ( v t ) = p ( t ) v t for all t ∈ [ T ] and the input at t ime t can be simplified as p ( t ) . As a direct application, one ca n obtain from Sec. 6 that the u p per bound for the one-way t rading problem is ln θ + 1, which matches the lower b ound. Thus, we immediately know that the op timal competitive ratio fo r one-way trading is l n θ + 1 and CR-Pursuit ( ln θ + 1 ) is an optimal deterministic online algorithm. In this section, with the aim of demonstrating t he po ssibility of mathematically charac- terizing Φ ∆ ( π ) in sp ecific problems, we first de rive a closed -form expres sion of Φ ∆ ( π ) , then we obtain the o ptimal comp etitive ratio π ∗ by solving the characteristic e q uation Φ ∆ ( π ) = ∆ . In the classic one-way trading problem, given a ny input up to time t , denote d as σ [ 1: t ] , { p ( 1 ) , p ( 2 ) , . . ., p ( t )} , the opt imal offline re venue can be expressed as η O P T ( σ [ 1: t ] ) = ∆ · max σ [ 1: t ] . Giv en any π ≥ 1, we focus on CR -Pursuit ( π ) d efined in Sec. 5. At time t , CR-Pursuit ( π ) sells the amount ¯ v t ∈ [ 0 , ∆ ] t hat satisfies: ¯ v t = 1 π · p ( t ) h η O P T  σ [ 1: t ]  − η O P T  σ [ 1: t − 1 ]  i . (15) As discussed, CR-Pursuit ( π ) aims at keeping the offline- to-onli ne rev en ue r ati o to b e π > 1 at all time. From Sec. 5, we know that if Φ ∆ ( π ) ≤ ∆ , then CR-Pursuit ( π ) is feasible and it i s π -competitive. In the following, our goal i s to derive a close-form expression of Φ ∆ ( π ) . Observe that at slot t , t he selling decision of CR-Pursuit ( π ∗ ) can be simplified as ¯ v t = ∆ π ∗ · p ( t )  max σ [ 1: t ] − max σ [ 1: t − 1 ]  . (16) This suggests that CR-Pursuit ( π ∗ ) will sell only when the current price is higher than the best price so far . With this observatio n, we have the following lemma. Lemma 12. For CR-Pursuit ( π ) with π ≥ 1 , given any i nput σ [ 1: T ] , to compute Φ ∆ ( π ) , it is sufficient to co nsider increa sing-pri ce seq uences. Lemma 12 is a corollar y of Lemma 7 in that while t he marginal prices are determined b y the participation of the algorithm in the latter , it is constant here in the c l assic one- way trading problem. Lemma 12 can b e proved by observing that the re venue of both the offline and online algorithms remai n unchanged if the cur rent price is not the highest price so far , and in that case remo ving this price from the input sequence will not affect t he b ehaviors o f b oth the offline and online algorithms. From Lemma 12, we know t hat it is sufficient to consider the following increasing price sequence with length n ≤ T : m ≤ p 1 < p 2 < · · · < p n ≤ M . (17) Under the g iven price s equence, the o ptimal offline r ev enue at time t ∈ [ n ] can b e simplified as η O P T  σ [ 1: t ]  = p t ∆ . According t o (16), the outp ut of CR-Pursuit ( π ) is given by ¯ v t = ∆ π p t − p t − 1 p t , ∀ t ∈ [ n ] , where p 0 = 0. Then we have Φ ∆ ( π ) = max p 1 , p 2 , · · · , p n n Õ t = 1 ¯ v t = max p 1 , p 2 , · · · , p n ∆ π  1 + p 2 − p 1 p 2 + · · · + p n − p n − 1 p n  ( a ) = ∆ π  1 + ∫ M m 1 x d x  = ∆ π ( 1 + ln θ ) , where (a) holds when the i nput sequence in (17) satisfies n → ∞ and p i → p i + 1 , ∀ i ∈ [ n − 1 ] . Indeed, this is the w orst cas e input sequence for one-way trading pr oblem, also kn own as the “ cr it ical” input seq uence. Setting Φ ∆ ( π ) = ∆ yields t he sol ution that π ∗ = ln θ + 1. Consequently , we have the following result. Proposition 13. Wi t h π ∗ = ln θ + 1 , CR-Pursuit ( π ∗ ) is feasible and an optimal online algorithm for t h e one-way trading problem. The pro of fo llows the same idea and ap p roach as that of Theo - rem 6. W e leave it as an exercise for readers. Competitive Online Optimization under Inventory Constraints A C M Sigmetrics conference, June 2019, P hoenix, AZ USA 7.2 One-way T rading with Price Elasticity In this subsection, we consider the one-way trading problem in a generalized setting w ith an additional fle x ibility on the price mo del playing t he role of price elasticity . W e assume that price is affected by the t otal quantity sold at e ach sl o t, implying that the decision of how much t o sell affects the trading price, usually known in the economics literature as p rice elasticity . Specifically , we assume that at each slot t ∈ [ T ] , the price elas- ticity , defined as , f t ( v ) , is a conve x non-negativ e function of the selling quantity with f ( 0 ) = 0. Under this setting, the rev enue func- tion at t ime t be co mes д t ( v ) = ( p ( t ) − f t ( v )) v . This setting can be considered as a special case of OOI C and t he in put at time t can be simplifie d as ( p ( t ) , f t ( v )) . Here we have д ′ t ( 0 ) = p ( t ) ∈ [ m , M ] and f t ( v ) ∈ [ 0 , + ∞) , ∀ v ∈ [ 0 , ∆ ] , f t ( 0 ) = 0. Namely , the set of all possible rev enue functions can be expressed as G = { д t ( v ) | д t ( v ) = ( p ( t ) − f t ( v )) v , p ( t ) ∈ [ m , M ] , f t ( v ) ∈ [ 0 , + ∞ ) , ∀ v ∈ [ 0 , ∆ ] , f t ( 0 ) = 0 } . Note that when f t ( v ) = 0 , ∀ t ∈ [ T ] , the problem reduces to one- way t r ading problem co nsidered in Sec. 7.1. Thus we note that any deterministic online algorithm in one-way trading with price elas- ticity has a comp etitive ratio of at least ln θ + 1. When f t ( v ) = α t v , α t ≥ 0 , ∀ t ∈ [ T ] , the p roblem be comes a o ne-way tr ading problem with linear price elasticity , which is a common setting in economic literatu re, e.g. , in Cournot competition [46]. Consider the online algorithm CR-Pursuit ( π ) defined in Sec. 5 . When there is price elasticity in the setting, it is difficult to obtain the closed-form expression of Φ ∆ ( π ) . W e follow the analysis in Sec. 6 to obt ain an upper bou nd on Φ ∆ ( π ) . In particular , restating Lemma 9 under the parametric assump- tions of д t ( v t ) in t he one -way trading pr oblem with p rice elasticity , we can upper bound the selling q uantity o f CR-Pursuit ( π ) at each slot with a better characterization of c , r eflected in the followin g lemma. Lemma 14. For any input sequen ce σ , we have ¯ v t ≤ c д t ( ¯ v t ) p ( t ) , ∀ t ∈ [ T ] , where c = 2  1 + p 1 − 1 / π  − 1 . (18) W e note that value of c given in (18) is smaller than t hat derived in Lemma 9. The idea of the pro of in App endix A.10 is similar t o that of Lemma 9, but w e fur ther utilize the special structure o f д t (· ) here (i.e., the convexity of f t (· ) ). The tighter characterization of c allows us to de velop an online algorithm with b etter comp etitive ratio as compared t o the one obtained as a result of Sec. 6. Lemma 15. For CR-Pursuit ( π ) with π ≥ 1 , we have Φ ∆ ( π ) ≤ ¯ Φ ( π ) , where ¯ Φ ( π ) , 2 ∆ h π  1 + p 1 − 1 / π  i − 1 ( ln θ + 1 ) . Lemma 15 shows that Φ ∆ ( π ) is upper bounde d by ¯ Φ ( π ) . It is easy to show that ¯ Φ ( π ) is decreasing in π ≥ 1. Thus by setting ¯ Φ ( ¯ π ) = ∆ , we can guarante e that CR-P ursuit ( ¯ π ) is feasible. Then we have the following result, which shows t hat the co mpetitive ratio of CR-Purs uit ( ¯ π ) is ln θ + Ω ( 1 ) . Theorem 16 . Let ¯ π = ( ln θ + 1 ) 2 /( ln θ + 3 / 4 ) < l n θ + 4 / 3 . The online algorithm CR-Pursuit ( ¯ π ) is feasible and i s thus ¯ π -competi tive. Proof. With ¯ π = ( ln θ + 1 ) 2 /( ln θ + 3 / 4 ) , we have ¯ Φ ( ¯ π ) = ∆ . From Lemma 15, we know that Φ ∆ ( ¯ π ) ≤ ¯ Φ ( ¯ π ) = ∆ . Thus the results are immediate.  Note that ¯ π < ln θ + 4 / 3, w hich is very close to the lower bound of ln θ + 1. It also improves beyond the result of 2 ( ln θ + 1 ) if we follow t he characterization in Sec. 6; t he improvement is b ecause of the tighter b ound through Lemma 14. 8 BEY OND THE W ORST CASE MEN T ALI T Y Our CR-Pursuit frame work focuses only on achieving comp etitive- ness under the worst case inputs. This may limit its applications as worst case inputs or situ at ions may seldom occur in pr actice. Intu- itively , a “bett er" online algorithm would sell more of its inv entor y when the incoming rev enue function is “not adversarial” , i.e ., being more opportunistic. By design , CR-Purs uit i s pessimistic: it only maintains a fixed co mpetitive ratio π ∗ for the whole t rading pe- riod, eve n if some inputs are not adversarial. One way to improve the performance of CR- Pursuit for non-adversarial cases is as fol- lows: instead of trying to keep the comp etitive ratio as π ∗ during the whole p eriod, t he online algor ithm adaptively cho oses a π t to maintain at time t . T his π t is chosen as the smallest, yet att ainable, competitive ratio at time t , given the previ ous inputs and out puts of the algorithm, and taking into account t he p ossible inputs in future slots. This app roach allows an online algorithm to instead pursue a comp etitive ratio more adaptive to the inputs, improving its average -case p erformance. W e have recently applied t he idea to dev elop online ele ctric v ehicle chargin g algorithms w it h o ptimal worst case and uniquely strong average-case perfor mance [55]. T o better illustrate this idea, consider the following example of one-way tr ading. Let t he first price b e p ( 1 ) = M . T he original CR- Pursuit algorithm sells an ∆ /( ln θ + 1 ) amount of i nven tory , and is satisfied with pursuing such co mpetitive ratio at all time . The suggested algorithm in this section knows that the optimal offline value cannot increas e any further , and would therefore sell all the inve ntory , i.e ., we can set π 1 = 1. In this case , it will sell all the inve ntory in the first slot and achiev e an o ffline-to-online re venue ratio of 1 for the p articular input. 9 CONCLUDING RE MARKS Online optimization is an impo rtant line o f resear ch with wide ranging applications. It has been tackled by multiple algorithmic approaches over the previous decades, each pro ving successful for differe nt problem variations, e .g., primal-dual approaches for on- line covering and packing problems or p otential functions for t he k -server problem. In this work, we presen t a novel al gor ithmic framework for on- line optimization with inven tory constraints. The framework “pur- sues” a bound on the comp etitive ratio, tracking the changes in the offline optimal algorithm and ensuring that the offline-to-online re venue ratio for the instance remains bounde d throughout the entire perio d. This idea all ows us to provide an nearly opt imal al- gorithm for online optimization with inve ntory constraints as well A C M Sigmetrics conference, June 2019, P hoenix, AZ USA Q . Lin, H. Yi, J. Pang, M. Chen, A. Wierman, M. Honig, and Y . Xiao as generalizations of the cl assical one- way trading problem. Specif- ically , our analysis and algorithms generalize naturally t o one-way trading problems w ith price elasticity and concave rev enue func- tions, yi elding almost opt imal (in terms of co mpetitive rat io) online algorithms in those settings. While our fo cus in this paper is on settings where inventory cannot b e replenished, ther e is a wide range of appl ications with both selling period s and buying period s, like battery arbitrage in contingency markets. Usually in these markets, prices are hi ghly affected by the selling quantities an d also other factors that vary in time, which lead to unknown incoming rev enue f unctions. W e believe that the CR-Pursuit framework is promising for these prob- lems as well, and can p otentially b e applicable to much broader classes of online o p timization problems. For example , our fo cus in this paper has been on worst case anal- ysis but the CR-Pursuit framew ork can also be used to provide “be- yond worst case ” results by parameterizing the bo und in different ways by , for example, ut ilizing prop erties of t he instances rele vant to the application, and adaptively considering the input seen so far; see a recent e xample in [55]. A dditionally , the framework can make use of ran domization when pursuing the CR bound. This may al- low improv ement beyond the deterministic lowe r bou nd discussed in this p aper , although it is an open question whether r andomized algorithms can outp erform deterministic algorithms for OOIC. 10 AC KNO WLEDGMEN TS W e acknowledge the supp ort receiv e d from H ong Kong Uni versity Grants Committee T heme-based Resear ch Scheme Project No. T23- 407/13-N and Coll aborative Research Fund No. C7036 -15 G, NSF grant AST -134338, NSF AitF-1637598, NSF CNS-15189 41, and NSF CPS-154471. In particular , John Pang wants to acknowledge t he suppor t from AST AR, Singapore. REFERENCES [1] Jacob D Ab ernethy , E lad Haz an, and A lexander Rakhlin. 2009. Competing in th e dark: An efficient algorithm for bandit l inear optimization. (2009). [2] H. Akhava n-Hejazi and H. Mohsenian-Rad. 2014. O ptimal Operation of Inde- pendent Storage Systems i n En ergy and Reserve Markets With High Wind Pen- etration. IEEE Transactions on Sm art Grid 5, 2 (March 2014), 1088–1097. [3] Susanne Albers. 2003. Online algorithms: a survey. Mathematical Programm ing 97, 1-2 (2003), 3–26. [4] Lachlan Andrew , Siddharth Barman, K atrina Ligett, Minghong Lin, Adam Mey- erson, Al an R oytman, and Adam Wierman. 2013. A tale of two metrics: Simulta- neous bounds on competitiveness and regret. In Conference on Learning Theory . 741–763. [5] Antonios Antoniadis, N eal Barcelo, Mic ha el Nugent, Kirk Pruhs, Kevin Schewior, and Michele Scquizza to. 2016. Chasing convex bodies and functions. In Latin A m erican Symposium on Theoretical Informatics . Spri nger , 68–81. [6] Sanjeev Arora, Elad Hazan, and Satyen Kale. 2012. T he M ultiplicative W eights Update Method: a Meta- Algorithm and Applications. Theory of Computing 8, 1 (2012), 121–164. [7] Peter A uer , Nicolo Cesa-Bi a nchi, Y oav Freund, and Rob ert E Schapire. 1995. G a m- bling in a rigged casino: The adversarial multi-armed bandit problem. In IEE E FOCS . 322. [8] Y . Azar, N. Buchbinder , T . H. Chan, S. Chen, I. R. Cohen , A. Gupta, Z. H uang, N. Kang, V . Nagarajan, J. Naor, a nd D . Panigrahi. 2016. Online Alg orithms for C ov- ering and Pac k ing Problems with Convex O bjectives. In 2016 IEE E 57th Annua l Symposium on Foundations of Computer Science (FOCS) . 148–157. [9] Moshe Babaioff, Nicole Immorlica , Davi d Kempe, and Robert Kleinberg. 2008. Online auctions and genera lized secretar y problems. ACM SIGecom Exchanges 7, 2 (2008). [10] Nikhil Bansa, Martin Böhm, Marek Eliáš, Gri gorios Koumoutsos, and Seeun William Umboh. 2018. Nested convex bodies ar e chasea ble. In Proceed- ings of the Twe nty-Ninth A nnual A CM-SIAM Symposium on Discrete Algorithms . 1253–1260. [11] Nikhil Bansa l, Niv Buchbinder, and Joseph Seffi Naor. 2012. A primal-dual ra n- domized a lgorithm for weighted paging. Journal of the ACM ( JA CM ) 59, 4 (2012). [12] Nikhil Bansal, Anupam Gupta, Ravishankar Krishnaswamy, Kirk Pruhs, Kevin Schewior , and Cliff Stein. 2015. A 2- competitive algorithm for online convex optimization with switching costs. In LIPIcs-Leibniz International Proceedings in Informatics , V ol. 40. Schloss Dagstuhl-Leibniz-Zentrum fuer In formatik. [13] H. Bogucka , M. Parz y , P. Marques, J. W . Mwangoka, and T . Forde. 2012. Sec- ondary spectrum trading in TV white s paces. IE EE C ommunications Magazine 50, 11 (November 2012), 121–129. [14] Allan Borodin, Nathan Linia l, and Micha el E Saks. 1992. An optimal on-line algorithm for metri cal task system. Journal of the ACM ( JA CM ) 39, 4 ( 1992), 745–763. [15] Sébastien Bubeck, N icolo Ce s a-Bianchi, et al. 2012. Regret analysis of stochastic and nonstochastic multi-a rmed bandit problems. Foundations and Trends® in Machine Learning 5, 1 (2012), 1–122. [16] Niv Buchbinder and Joseph Naor . 2005. Online primal-dual algorithms for cov- ering and packing problems. In European Symposium on Algorithms . Spri nger , 689–701. [17] Niv Buchbinder , Joseph Seffi Naor , et al. 2009. The design of competitive online algorithms via a primal–dual approach. Foundations and Trends® in T heoretical Computer Science 3, 2–3 (2009), 93–263. [18] Niangjun Chen, Anish Agarwal, Adam Wierm a n, Siddharth Barman, and Lach- lan LH Andrew . 2015. Online convex optimization using predictions. In A CM SIGMETRICS Performance Evaluation Review , V ol. 43. 191–204. [19] Francis YL Chin, Bin Fu, Jiuling Guo, Shugua ng Han, J u eliang Hu, Minghui Jiang, Guohui Lin, Hing-Fung Ting, L uping Zhang, Y ong Zhang, et al. 2015. C ompet- itive algorithms for unbounded one-way trading. Theoretical Com puter Science 607 (2015), 35–48. [20] YS Chow , Sigaiti Moriguti, Herb ert Robbins, and SM Samuels. 1964. Optimal se- lection bas ed on relative r ank (the “secretar y problem”). Israel Journal of math- ematics 2, 2 (1964), 81–90. [21] Peter D amaschke, P huong Hoai Ha, and P hilippas Tsigas. 2009. Online search with time-varying price bounds. Algorithmica 55, 4 (2009), 619–642. [22] Nikhil R. Devanur and Kamal Ja in. 2012. Online Matching with Concave Re- turns. In Proceedings of the Forty-fourth Ann ual ACM Symposium on Theor y of Computing (STOC ’ 12) . A C M, 137–144. [23] Ran El- Y aniv, Amos Fiat, Richard M Karp , and Gordon Turpin. 2001. Optimal search and one-way trading onl ine algorithms. Algorithmica 30, 1 (2001), 101– 139. [24] Moran Feldman a nd Rico Zenklusen. 2018. The submodular secre ta r y problem goes linear . SIAM J. Comput. 47, 2 (2018), 330–366. [25] Amos Fiat. 1998. Online Algorithms: The State of the Art (Lec tur e N otes i n Computer Science). (1998). [26] Amos Fia t and Ma nor Mendel. 2003. Better algorithms for unfair metrica l task systems and applications. SIAM J. Comput. 32, 6 (2003), 1403–1422. [27] Amos Fiat, Yuval Rabani, and Yiftach Ravid. 1990. Competitive k-s erver algo- rithms. (1990). [28] Joel Friedma n and Nathan Linial. 1993. On convex b ody chasing. Discrete & Computational Geometr y 9, 3 (1993), 293–321. [29] Hiroshi Fujiwara, Kazuo Iwama, a nd Y oshiyuk i Sekiguchi. 2011. A ver age-case competitive analyses for one-way trading. Journal of Combinatorial Optim iza- tion 21, 1 (2011), 83–107. [30] Edward F G r ove. 1991. The harmonic online k- server a lgorithm is competitive. In Proceedings of the twenty-third annu al ACM symposium on T heory of computing . 260–266. [31] Linqi Guo, John Pang, and Anwar W alid. [n. d.]. Dynamic ser vice function cha i n- ing in SDN -enabled netw orks with middl eboxes. In 2016 IEE E 24t h International Conference on Network Protocols (ICNP) . 1–10. [32] Linqi Guo, John Pang, and Anwar Walid. 2018. Joint Placement and Routing of Network Function Chains in Data Centers. In IE E E INFOCOM . 612–620. [33] Mohammad T aghi Hajiaghayi, Rob ert Kleinberg, and T uomas Sandholm. 2007. A utomated onl ine m echanism design and prophet i nequalities. In AAAI , V ol. 7. 58–65. [34] Elad Haza n, Amit Agarwal, and Satyen K a le. 2007. L ogarithmic regret algo- rithms for online convex optimization. Machine Learning 69, 2- 3 (2007), 169– 192. [35] Sham M Kakade, M ichael K earns, Yisha y Mansour , and Luis E Ortiz. 2004. Com- petitive a lgorithms for VW AP a nd limit order tra ding. In Proceedings of A CM EC . 189–198. [36] Anna R Karlin, Ma rk S Manas s e, Larr y Rudolph, and Daniel D Sleator . 1988. Com- petitive snoopy caching. Algorithmica 3, 1-4 (1988), 79–119. [37] Jiayin Li, Meikang Qiu, Zhong Ming, Gang Quan, Xiao Qin, and Zonghua Gu. 2012. Online optimiza tion for scheduling preemptab le tasks on Ia aS cloud sys- tems. J. Parallel and Distrib. Comput. 72, 5 (2012), 666–677. [38] Yingying Li, Guannan Qu, and Na Li. 2018. Using Predictions in Online Opti- mization with Switching Costs: A Fas t Algorithm a nd A Fundamental Limit. In IEEE A nnual A merican Control Conference (A CC) . 3008–3013. Competitive Online Optimization under Inventory Constraints A C M Sigmetrics conference, June 2019, P hoenix, AZ USA [39] Minghong Lin, Adam Wierm a n, Lachlan LH Andrew , and Eno Thereska. 2013. Dynamic r ight-sizing for power-pr oportional data centers. IEEE /ACM T ransac- tions on Networking (TON) 21, 5 (2013), 1378–1391. [40] Minghong Lin, Ad am Wierman, Alan Roytman, A dam Meyerson, and Lach- lan LH Andrew . 2012. Online optimization with s witching cost. ACM SIGMET - RICS Performance Evaluation Rev iew 40, 3 (2012), 98–100. [41] L.Lu , J. Tu, C.Chau, M.Chen, and X.Lin. 2013. Online Energy Generation Sched- uling for M icrogrids w ith Intermittent Energy Sources and Co-Generation. Pro- ceedings of ACM Sigm etrics (2013). [42] Julian Lorenz, Konstantin os P anagiotou, and Angelika Steger . 2009. Optimal algorithms for k-search with application in option pricing. Algorithmic a 55, 2 (2009), 311–328. [43] Tan Lu, Minghua Chen, a nd La chlan LH Andrew . 2013. Simple and effective dynamic provisioning for power-proportional data centers. IEEE Transactions on Parallel and Distributed Systems 24, 6 (2013), 1161–1171. [44] Jacob Mattingley , Y ang Wang, and Steph en Boyd. 2011. Receding horizon con- trol. IEEE Control S ystems 31, 3 (2011), 52–65. [45] Hanna Michalska and Dav i d Q Mayne. 1993. Robust r eceding horizon cont rol of constrained nonlinear sys tems. IEEE transactions on automatic control 38, 11 (1993), 1623–1633. [46] John ZF Pang, Hu Fu, W on I L ee, and Adam Wierman. 2017. The Efficiency of Open Access in Platforms for Netwo rked Cournot Markets. In Proceedings of IEEE INFOCOM . [47] John ZF Pang, Linqi Guo, and Steven H Low . 2017. Optimal load control for frequency regulation under limited con trol coverage. In IR EP Symposium . 1–7. [48] L. Qian, F . Y e, L. Gao, X. Gan, T . Chu, X. Tian, X. Wang, and M. Guizani. 2011. Spectrum Trading in Cognitive Radio N etworks: An Agen t-Based Model under Demand Uncertainty. IEE E Transactions on Comm unications 59, 11 (November 2011), 3192–3203. [49] Xiaoqi Ren, Palma London, Juba Ziani, and Adam W ierman. 2018. Datum: Man- aging Data Purcha sing and Data Placement in a Geo-Distri buted D ata Market. IEEE/ACM Transact ions on Networking 26, 2 (2018), 893–905. [50] A via d Rubinstein. 2016. Beyond matroids: Secretar y problem a nd prophet in- equality w ith general constraints. In Proceedings of the forty-eighth annu al A CM symposium on Theory of Computing . 324–332. [51] Karl H Schlag. 1998. Why imitate, and if so, how?: A boundedly r ational ap- proach to multi-armed bandits. Journal of Economic Theory 78, 1 (1998), 130– 156. [52] S. Shafiee, A. Bhuiya, A. U. Haque, H. Zar eipour , and A. M. Knight. 2018. Eco- nomic Assessment of Energy Storage Systems in Alberta’s Energy and Operat- ing Reserve Mark ets. In 2018 IEEE/PES Transmiss ion and Distribution Conference and Exposition . 1–5. [53] Shai Shalev-Shwartz and Sham M Kakade. 2009. Mind t he dua lity gap: Logarith- mic regret algorithms for onlin e optimization. In Advances in N eural Information Processing Systems . 1457–1464. [54] Lin Y ang, Mohammad H Hajiesmaili, H anling Yi, and Minghua C hen. J une 2017. Online Offering Strategies for Storage-Assisted Renewable Pow er Producer in Hour-A head Market. In Proceedings of A CM SIGMETRICS . [55] Hanling Yi* , Qiulin Lin* , and Minghua Chen. 2019. Balancing Cost and Dissa tis- faction in O nline EV Charging under Rea l-time Pricing. In IEEE INFOCOM . (The first two authors cont ribute equally to the work.). [56] W enming Zh ang, Yinfeng Xu, Feifeng Zheng, and Yucheng Dong. 2012. Optimal algorithms for online time series search and one-way trading with interrelated prices. Journal of combinatorial optimization 23, 2 (2012) , 159–166. [57] Ying Zhang, Mohamma d H Hajiesmaili, Sinan Cai, Minghua Chen, and Qi Zhu. 2018. P eak-aware online economic dispatching for micr ogrids. IEEE Transactions on Smart Grid 9, 1 ( 2018), 323–335. A APPENDIX A.1 A Bi nary Search Algorithm for Computing λ ∗ W e summarize the algorithm in Algorithm 2. A.2 Proof of Proposition 1 Proof. W e prov e t his theorem by investigating the KKT condi- tions of problem OOIC and exploring the st ructure of t he optimal solution. The Lagrangian for problem OOIC is defined as L ( v , λ , µ ) = T Õ t = 1 д t ( v t ) + λ ∆ − T Õ t = 1 v t ! + T Õ t = 1 v t µ ( t ) , Algorithm 2 A Binary search algorithm for Computing λ ∗ 1: if max v t ∈ V t ( 0 ) Í T t = 1 v t ≤ ∆ then 2: return λ ∗ = 0; 3: else 4: Pick λ L = 0 , λ H = max t ∈ T  д ′ t ( 0 )  ; 5: while | λ L − λ H | > ϵ do 6: λ M = λ L + λ H 2 , v t = 0 , ∀ t ∈ T ; 7: Compute ∆ max = max v t ∈ V t ( λ M ) Í T t = 1 v t 8: Compute ∆ min = min v t ∈ V t ( λ M ) Í T t = 1 v t . 9: if ∆ min > ∆ then 10: λ L = λ M ; 11: end if 12: if ∆ max < ∆ then 13: λ H = λ M ; 14: end if 15: if ∆ min ≤ ∆ ≤ ∆ max then 16: break; 17: end if 18: end while 19: return λ ∗ = λ M ; 20: end if where λ ≥ 0 and µ ( t ) ≥ 0, ∀ t ∈ [ T ] ar e the Lagrang ian multipli- ers. The foll owing KKT conditions give us a se t of n ecessary an d sufficient conditions for optimality: д ′ t ( v t ) − λ + µ ( t ) = 0 , ∀ t ∈ [ T ] , T Õ t = 1 v t ≤ ∆ , v t ≥ 0 , ∀ t ∈ [ T ] , µ ( t ) ≥ 0 , ∀ t ∈ [ T ] , λ ≥ 0 , v t µ ( t ) = 0 , ∀ t ∈ [ T ] , λ T Õ t = 1 v t − ∆ ! = 0 . Suppose v ∗ , µ ∗ and λ ∗ are the optimal sol u tions that satisfy t he KKT conditions. Denote the set T 0 = { t | v ∗ t > 0 , ∀ t ∈ [ T ] } , then according to the KKT conditions, we have µ ∗ ( t ) = 0 , ∀ t ∈ T 0 , (19) λ ∗ ©  « Õ t ∈ T 0 v ∗ t − ∆ ª ® ¬ = 0 , (20) д ′ t ( v ∗ t ) − λ ∗ = 0 , ∀ t ∈ T 0 , (21) Since д ′ t is concave , д ′ t (· ) is non-incr easing in v t . Accordin g to (21) we have д ′ t ( 0 ) ≥ д ′ t ( v ∗ t ) = λ ∗ , ∀ t ∈ T 0 ; namely , д ′ t ( 0 ) ≥ λ ∗ ∀ t ∈ T 0 . (22) Thus given a λ ∗ , we can use (22) to determine the set T 0 . A C M Sigmetrics conference, June 2019, P hoenix, AZ USA Q . Lin, H. Yi, J. Pang, M. Chen, A. Wierman, M. Honig, and Y . Xiao For ease of presentation, we denote V t ( λ ) = { v | д ′ t ( v ) = λ , v ∈ [ 0 , ∆ ] } . Now consider the following two cases: (1) ∆ ≥ max v t ∈ V t ( 0 ) Í T t = 1 v t . In this case, we observe that t he solution v ∗ t ∈ V t ( 0 ) , ∀ t ∈ [ T ] , λ ∗ = 0 , µ ∗ ( t ) = 0 , ∀ t ∈ [ T ] , satisfies the KK T conditions, thus it is the opt imal solution. (2) ∆ < max v t ∈ V t ( 0 ) Í T t = 1 v t . In this case, we must have λ ∗ > 0. According t o (20) and (21), we have v ∗ t ∈ V t ( λ ∗ ) and T Õ t = 1 v ∗ t = ∆ . It is straightforw ard to check that v t , ∀ t ∈ T 0 , is non-increasi ng w .r .t. λ . M eanwhile, according t o (22), we know that the size of set T 0 is non-increasing w. r .t. λ . Putting to gether these two ob - servations, we conclude that Í t ∈ T 0 v t is non-incr easing w .r .t. λ . Thus given ∆ > 0, there exists a unique λ = λ ∗ that satisfies Í t ∈ T 0 v ∗ t = ∆ . Since K KT conditions are necessar y and sufficien t for optimality of co nvex problems, we can conclude t hat λ ∗ is the optimal dual solut ion.  A.3 Proof of Lemma 2 Proof of Lemma 2. W e prov e this lemma in the fo l lowing two steps: Step I, we prove that η O P T  σ [ 1: t ]  − η O P T  σ [ 1: t − 1 ]  ≥ д t ( ˜ v t ) − λ t ˜ v t . T o see this, we denote optimal so l ution at time τ ∈ [ t ] under input σ [ 1: t ] as ˜ v τ . Note that ˜ v τ ∈ V τ ( λ t ) , τ ∈ [ t ] or ˜ v τ = 0 if V τ ( λ t ) = ∅ . Similarly , den ote optimal sol ution at time τ ∈ [ t − 1 ] under input σ [ 1: t − 1 ] as ¯ v τ . Note that ¯ v τ ∈ V τ ( λ t − 1 ) , τ ∈ [ t − 1 ] or ¯ v τ = 0 if V τ ( λ t − 1 ) = ∅ . Also ˜ v τ ≤ ¯ v τ , τ ∈ [ t − 1 ] (by the non-incr easing of д ′ t ( v ) and λ t ≥ λ t − 1 ). Then we have η O P T  σ [ 1: t ]  − η O P T  σ [ 1: t − 1 ]  = t Õ τ = 1 д τ ( ˜ v τ ) − t − 1 Õ τ = 1 д τ ( ¯ v τ ) = д t ( ˜ v t ) + t − 1 Õ τ = 1 ( д τ ( ˜ v τ ) − д τ ( ¯ v τ )) ( a ) ≥ д t ( ˜ v t ) + t − 1 Õ τ = 1 λ t ( ˜ v τ − ¯ v τ ) ( b ) ≥ д t ( ˜ v t ) − λ t ˜ v t . For ( a), it comes from the concavity of д τ ( v ) and ˜ v τ ≤ ¯ v τ , τ ∈ [ t − 1 ] . For (b), we claim that Í t − 1 τ = 1 ¯ v τ ≤ Í t τ = 1 ˜ v τ . T o se e this, when λ t = 0, we must have λ t − 1 = 0. In this case, ˜ v τ = ¯ v τ , ∀ τ ∈ [ t − 1 ] and thus we hav e Í t − 1 τ = 1 ¯ v τ ≤ Í t τ = 1 ˜ v τ . When λ t > 0 , from the KKT conditions in (20), we hav e Í t τ = 1 ˜ v τ = ∆ ≥ Í t − 1 τ = 1 ¯ v τ . Then we conclude t hat Í t − 1 τ = 1 ¯ v τ ≤ Í t τ = 1 ˜ v τ and consequ ently , we have Í t − 1 τ = 1 ( ˜ v τ − ¯ v τ ) ≥ − ˜ v t . Step II , we prove that η O P T  σ [ 1: t ]  − η O P T  σ [ 1: t − 1 ]  ≤ д t ( ˜ v t ) − λ t − 1 ˜ v t ≤ д t ( ˜ v t ) ≤ д t ( ˆ v t ) . Similarly , we have η O P T  σ [ 1: t ]  − η O P T  σ [ 1: t − 1 ]  = д t ( ˜ v t ) + t − 1 Õ τ = 1 ( д τ ( ˜ v τ ) − д τ ( ¯ v τ )) ( a ) ≤ д t ( ˜ v t ) + t − 1 Õ τ = 1 λ t − 1 ( ˜ v τ − ¯ v τ ) ( b ) = д t ( ˜ v t ) − λ t − 1 ˜ v t ≤ д t ( ˜ v t ) ≤ д t ( ˆ v t ) . For (a) , it is by the concavity of д τ : д τ ( ˜ v τ ) ≤ д τ ( ¯ v τ ) + λ τ − 1 ( ˜ v τ − ¯ v τ ) (Note that λ τ = д ′ τ ( ¯ v τ ) ) and λ t − 1 ≥ λ τ , ∀ τ ∈ [ t − 1 ] . For (b), when λ t − 1 = 0, it holds immediately; when λ t − 1 > 0, we have Í t τ ˜ v τ = ∆ = Í t − 1 τ = 1 ¯ v τ , which implies Í t − 1 τ = 1 ( ˜ v τ − ¯ v τ ) = − ˜ v t .  A.4 Proof of Lemma 3 Proof. Denote the input under ˜ σ as д t . Denote the input under ¯ σ as ¯ д t , The opt imal dual variable under ˜ σ [ 1: t ] (resp . ¯ σ [ 1: t ] ) as λ t (resp . ¯ λ t ). W e have , д t = ¯ д t , ∀ t ≤ τ − 1 ∨ t ≥ τ + 2 . Besides, д τ = ¯ д τ + 1 , д τ + 1 = ¯ д τ . Let v t (resp . ¯ v t ) b e the optimal offline solution at time t give n the input ˜ σ [ 1: t ] (resp . ¯ σ [ 1: t ] ). 1) If λ τ ≤ ¯ λ τ , then η O P T  ˜ σ [ 1: τ ]  − η O P T  ˜ σ [ 1: τ − 1 ]  ( a ) ≥ д τ ( v τ ) − λ τ v τ ( b ) ≥ д τ ( ¯ v τ + 1 ) − λ τ ¯ v τ + 1 ( c ) ≥ д τ ( ¯ v τ + 1 ) − ¯ λ τ ¯ v τ + 1 ( a ) ≥ η O P T  ¯ σ [ 1: τ + 1 ]  − η O P T  ¯ σ [ 1: τ ]  . For ( a), it is by lemma 2 . For ( b), it is by the concavity of д t and for (c), it by λ τ ≤ ¯ λ τ . 2) If λ τ ≥ ¯ λ τ , then similarly η O P T  ¯ σ [ 1: τ ]  − η O P T  ¯ σ [ 1: τ − 1 ]  ( a ) ≥ д τ + 1 ( ¯ v τ ) − ¯ λ τ ¯ v τ ( b ) ≥ д τ + 1 ( ¯ v τ ) − λ τ ¯ v τ ( c ) ≥ д τ + 1 ( v τ + 1 ) − λ τ v τ + 1 ( a ) ≥ η O P T  ˜ σ [ 1: τ + 1 ]  − η O P T  ˜ σ [ 1: τ ]  . For (a), it is by lemma 2. For ( b), it i s by λ τ ≥ ¯ λ τ . For ( c), it is by the concavity of д t . Also, with η O P T  ¯ σ [ 1: τ ]  − η O P T  ¯ σ [ 1: τ − 1 ]  + η O P T  ¯ σ [ 1: τ + 1 ]  − η O P T  ¯ σ [ 1: τ ]  = η O P T  ¯ σ [ 1: τ + 1 ]  − η O P T  ¯ σ [ 1: τ − 1 ]  = η O P T  ˜ σ [ 1: τ + 1 ]  − η O P T  ˜ σ [ 1: τ − 1 ]  = η O P T  ˜ σ [ 1: τ + 1 ]  − η O P T  ˜ σ [ 1: τ ]  + η O P T  ˜ σ [ 1: τ ]  − η O P T  ˜ σ [ 1: τ − 1 ]  , Competitive Online Optimization under Inventory Constraints A C M Sigmetrics conference, June 2019, P hoenix, AZ USA we can have η O P T  ¯ σ [ 1: τ + 1 ]  − η O P T  ¯ σ [ 1: τ ]  ≤ η O P T  ˜ σ [ 1: τ ]  − η O P T  ˜ σ [ 1: τ − 1 ]  .  A.5 Proof of Lemma 7 Proof of Lemma 7. Suppose an arbitrar y ˜ σ ∈ arg max σ Í t v t , under w hich д ′ t ( v t ) is not non-decreasing in t , where v t is the sell- ing quantity of CR-Pursuit( π ) under ˜ σ . That is, exist a τ , д ′ τ ( v τ ) > д ′ τ + 1 ( v τ + 1 ) . Denote the o ptimal dual variables un der ˜ σ [ 1: t ] as λ t . Note that λ t is non-decreasing in t . Without loss of generality , we assume that λ t < λ t + 1 or λ t = λ t + 1 = 0, ∀ t . W e construct a new input sequence ¯ σ by inter changing д τ and д τ + 1 in ˜ σ and denote the input under ¯ σ as ¯ д t , the outpu t of CR-Pursuit ( π ∗ ) under ¯ σ as ¯ v t . The o ptimal dual variable under ¯ σ [ 1: t ] as ¯ λ t . By definition, we can easily observe that, η O P T  ˜ σ [ 1: t ]  = η O P T  ¯ σ [ 1: t ]  , ∀ t ≤ τ − 1 ∨ t ≥ τ + 1; v t = ¯ v t , ∀ t ≤ τ − 1 ∨ t ≥ τ + 2; д t = ¯ д t , ∀ t ≤ τ − 1 ∨ t ≥ τ + 2 . Besides, д τ = ¯ д τ + 1 , д τ + 1 = ¯ д τ . W e claim that ¯ σ ∈ arg max σ Í t v t and ¯ д ′ τ ( ¯ v τ ) = д ′ τ + 1 ( v τ + 1 ) < д ′ τ ( v τ ) = ¯ д ′ τ + 1 ( ¯ v τ + 1 ) . T o se e this, con- sider the following two cases. (1) λ τ = λ τ + 1 = 0. Under this case, we have η O P T  ˜ σ [ 1: τ ]  − η O P T  ˜ σ [ 1: τ − 1 ]  = η O P T  ¯ σ [ 1: τ + 1 ]  − η O P T  ¯ σ [ 1: τ ]  = д τ ( ˆ v τ ) , where ˆ v τ = arg max v д τ ( v ) . Then v τ = ¯ v τ + 1 . Similarly , w e have v τ + 1 = ¯ v τ . Í t v t = Í t ¯ v t . W e conclude t hat ¯ σ ∈ arg max σ Í t v t and ¯ д ′ τ ( ¯ v τ ) = д ′ τ + 1 ( v τ + 1 ) < д ′ τ ( v τ ) = ¯ д ′ τ + 1 ( ¯ v τ + 1 ) . (2) 0 ≤ λ τ < λ τ + 1 . First, we have д τ ( v τ ) + д τ + 1 ( v τ + 1 ) = η O P T  ˜ σ [ 1: τ + 1 ]  − η O P T  ˜ σ [ 1: τ − 1 ]  π ∗ = η O P T  ¯ σ [ 1: τ + 1 ]  − η O P T  ¯ σ [ 1: τ − 1 ]  π ∗ = д τ + 1 ( ¯ v τ ) + д τ ( ¯ v τ + 1 ) , which implies д τ ( v τ ) − д τ ( ¯ v τ + 1 ) = д τ + 1 ( ¯ v τ ) − д τ + 1 ( v τ + 1 ) . Second, we claim t hat ¯ v τ + 1 ≤ v τ . From Lemma 3, we have η O P T  ¯ σ [ 1: τ + 1 ]  − η O P T  ¯ σ [ 1: τ ]  ≤ η O P T  ˜ σ [ 1: τ ]  − η O P T  ˜ σ [ 1: τ − 1 ]  . Then д τ ( v τ ) ≥ д τ ( ¯ v τ + 1 ) and ¯ v τ + 1 ≤ v τ are straightforward. Third, we show д τ ( v τ ) = д τ ( ¯ v τ + 1 ) and thus ¯ v τ + 1 = v τ by contradiction. Sup pose д τ ( v τ ) > д τ ( ¯ v τ + 1 ) and thus ¯ v τ + 1 < v τ . we show that Í t v t < Í t ¯ v t which contradict the fact that ˜ σ ∈ arg max σ Í t v t . T o se e this, observe that we have д ′ τ + 1 ( v τ + 1 )( v τ − ¯ v τ + 1 ) ( a ) < − д ′ τ ( v τ )( ¯ v τ + 1 − v τ ) ( b ) ≤ д τ ( v τ ) − д τ ( ¯ v τ + 1 ) = д τ + 1 ( ¯ v τ ) − д τ + 1 ( v τ + 1 ) ( b ) ≤ д ′ τ + 1 ( v τ + 1 )( ¯ v τ − v τ + 1 ) . For (a), it is by д ′ τ ( v τ ) > д ′ τ + 1 ( v τ + 1 ) ≥ λ t + 1 > 0 and ¯ v τ + 1 < v τ . For ( b), it is from the co ncavity o f д τ . As д ′ τ + 1 ( v τ + 1 ) ≥ λ τ + 1 > 0, we have v τ + v τ + 1 < ¯ v τ + ¯ v τ + 1 , which leads to Í t v t < Í t ¯ v t . So we conclude that д τ ( v τ ) = д τ ( ¯ v τ + 1 ) and t hus ¯ v τ + 1 = v τ . Consequently , д τ ( v τ + 1 ) = д τ + 1 ( ¯ v τ ) and thus ¯ v τ = v τ + 1 . It is then straightforward that ¯ σ ∈ arg max σ Õ t v t , and ¯ д ′ τ ( ¯ v τ ) = д ′ τ + 1 ( v τ + 1 ) < д ′ τ ( v τ ) = ¯ д ′ τ + 1 ( ¯ v τ + 1 ) . By continuously interchangi ng д τ and д τ + 1 that fails to satisfy д ′ τ + 1 ( v τ ) ≤ д ′ τ ( v τ + 1 ) , we finally attain a se quence in arg max σ Í t v t such that д ′ t ( v t ) is non-decreasing in t .  A.6 Proof of Lemma 9 Proof. First, from Lemma 2, w e easily conclude that ¯ v t ≤ ˆ v t , where ˆ v t is t he optimizer of д t (· ) . By t he concavity o f д t (· ) , we have д t ( ¯ v t ) ≥ ¯ v t ˆ v t д t ( ˆ v t ) +  1 − ¯ v t ˆ v t  д t ( 0 ) ≥ ¯ v t ˆ v t д t ( ˆ v t ) , which t hen gives ¯ v t ≤ д t ( ¯ v t ) д t ( ˆ v t )/ ˆ v t . Then, using the definition of c , we arrive at ¯ v t ≤ p ( t ) д t ( ˆ v t ) / ˆ v t д t ( ¯ v t ) p ( t ) ≤ c д t ( ¯ v t ) p ( t ) .  A.7 Proof of Lemma 10 Proof. For ease of presentation, define x p , π ∆ Õ { t : p ( t ) ≤ p } д t ( ¯ v t ) . It is then equilivalent to show that x p ≤ p . Define T 1 , min { t : p ( τ ) > p , ∀ τ ≥ t } − 1, i. e., for any t > T 1 , we hav e p ( t ) > p , o r equiv- alently if p ( t ) ≤ p , then t ≤ T 1 . By definition, x p is determined by σ [ 1: T 1 ] only . Thus, in this proof, we only focus on the input horizon t ∈ [ T 1 ] . W e first consider a special case when p ( t ) ≤ p , ∀ t ∈ [ T 1 ] . By that д t ( v ) , ∀ t ∈ [ T 1 ] are concave functions, we have η O P T  σ [ 1: T 1 ]  = T 1 Õ t = 1 д t  v ∗ t  ≤ T 1 Õ t = 1  д t ( 0 ) + д ′ t ( 0 ) v ∗ t  = T 1 Õ t = 1 p ( t ) v ∗ t , A C M Sigmetrics conference, June 2019, P hoenix, AZ USA Q . Lin, H. Yi, J. Pang, M. Chen, A. Wierman, M. Honig, and Y . Xiao where v ∗ t , t ∈ [ T 1 ] are t he solut ion of the optimal offline algorithm under input σ [ 1: T 1 ] . T hen according to (9) and that η O P T  σ [ 1: T 1 ]  ≤ p · ∆ , we have x p = π ∆ T 1 Õ t = 1 д t ( ¯ v t ) = 1 ∆ η O P T  σ [ 1: T 1 ]  ≤ p . W e now consider the gene ral cases, where there could be some slot(s) τ ∈ [ T 1 ] such that p ( τ ) > p . The we co nstruct a new input se- quence ¯ σ by interchange д τ and д τ + 1 in σ . Denote the input under ¯ σ as ¯ д t . Let ¯ x p , ¯ p ( t ) be the corresponding variables under ¯ σ . T o show t hat x p ≤ p , we first show x p ≤ ¯ x p . By definition, we observe that, η O P T  σ t  = η O P T  ¯ σ t  , ∀ t ≤ τ − 1 ∨ t ≥ τ + 1; д t = ¯ д t , ∀ t ≤ τ − 1 ∨ t ≥ τ + 2 . Besides, д τ = ¯ д τ + 1 , д τ + 1 = ¯ д τ . W e discuss two cases. • When p ( τ + 1 ) > p : it is easy to see that ¯ x p = x p . W e t hen pro ve x p ≤ p as follows: W e continuously inter- change with p ( τ ) > p with the input at its next slot until all the slots with p ( t ) ≤ p is at t he front of it. At the meantime, x p keeps on non-decreasing. Finally , we get a σ ′ , in which the price at each slot in [ T ′ 1 ] ( T ′ 1 is corresponding t o T 1 but defined under σ ′ ) is less or e qual to p , and x p ≤ x ′ p . Since in σ ′ , p ≥ p ( t ) , ∀ t , from our analysis in the first p art (special case), we have x ′ p ≤ p . It then follows that x p ≤ p . • When p ( τ + 1 ) ≤ p : we have x p − ¯ x p = η O P T  σ [ 1: τ + 1 ]  − η O P T  σ [ 1: τ ]  ∆ − η O P T  ¯ σ [ 1: τ ]  − η O P T  ¯ σ [ 1: τ − 1 ]  ∆ ( a ) ≤ 0 , where the step (a) is b ecause of Lemma 3. Next, we pro ve x p ≤ p . W e continuously interchange w ith p ( τ ) > p with the input at its next slot until all the slots with p ( t ) ≤ p is at the front of it . At the meantime , x p keeps on non-decrea sing. Finally , w e ge t a σ ′ , in which the price at each slot in [ T ′ 1 ] ( T ′ 1 is corresponding to T 1 but d efined under σ ′ ) is less or eq ual to p , and x p ≤ x ′ p . Sin ce in σ ′ , p ≥ p ( t ) , ∀ t , from o ur analysis in the fir st part (special case), we have x ′ p ≤ p . It then follows that x p ≤ p .  A.8 Proof of Lemma 11 Proof. Supp ose in σ [ 1: T ] , p ( t ) takes n different values, which are denoted as m ≤ p 1 ≤ p 2 ≤ · · · · · · ≤ p n ≤ M . And define y i , Í t , p ( t ) = p i π ∆ д t ( ¯ v t ) . Note that we have T Õ t = 1 д t ( ¯ v t ) p ( t ) = ∆ π n Õ i = 1 y i p i . From Lemma 10, we have Í i j = 1 y j = x p i ≤ p i . Consider the foll owing optimization problem: max n Õ i = 1 y i p i s . t . i Õ j = 1 y j ≤ p i , i ∈ [ n ] y i ≥ 0 , i ∈ [ n ] . The KKT conditions are sufficient and necessar y conditions for o p- timality for the above convex problem. D enote µ i ≥ 0 , i ∈ [ n ] as the dual variables, then t he KKT conditions can be expre ssed as: 1 p i − n + 1 − i Õ j = 1 µ i = 0 , ∀ i ∈ [ n ] , (23) µ i ( p i − i Õ j = 1 y j ) = 0 , ∀ i ∈ [ n ] , (24) µ i ≥ 0 , ∀ i ∈ [ n ] , y i ≥ 0 , ∀ i ∈ [ n ] . From (23), we know that µ i > 0 for all i ∈ [ n ] . Thus from (24), we have p i − i Õ j = 1 y j = 0 , ∀ i ∈ [ n ] . Thus we know the optimal primal solution is y i = p i − p i − 1 , ∀ i ∈ [ n ] , where p 0 = 0. And the optimal objective value equals to Í n i = 1 p i − p i − 1 p i . So T Õ t = 1 д t ( ¯ v t ) p ( t ) = ∆ π n Õ i = 1 y i p i ≤ ∆ π n Õ i = 1 p i − p i − 1 p i = ∆ π p 1 p 1 + n Õ i = 2 p i − p i − 1 p i ! ≤ ∆ π  1 + ∫ p n p 1 1 x d x  ≤ ∆ π ( 1 + ln θ ) . This completes ou r proof.  A.9 Proof of Lemma 12 Proof. W e show that any input σ [ 1: T ] is equivalent to (in the sense that the behaviors of both offlin e algorithm an d the pr opose d online algorithm remain unchanged) an increasing price sequ ence as the following: m ≤ p 1 < p 2 < · · · < p n ≤ M , (25) where n ≤ T . Accor ding to (16), CR-Pursuit ( π ) will sell only when the current price is larger than t he highest price in histor y . Thus for any in put σ [ 1: T ] , we can delete the slots when CR-Pursuit ( π ) Competitive Online Optimization under Inventory Constraints A C M Sigmetrics conference, June 2019, P hoenix, AZ USA does not sell, and the output s of CR-Pursuit ( π ) is then equivalent to the resulting increasing price seq uence.  A.10 Proof of Lemma 14 Proof. By Lemma 2 an d definition of д t ( ¯ v t ) , we kno w that π д t ( ¯ v t ) = η O P T  σ [ 1: t  − η O P T  σ [ 1: t − 1 ]  ≤ д t ( ˆ v t ) and then ¯ v t ≤ ˆ v t , where ˆ v t as the optimizer of д t ( v t ) . T o simplify the explanation, let k = д t ( ¯ v t ) and α = f ( ˆ v t ) ˆ v t . Define ˜ д t ( v t ) = ( p ( t ) − α v t ) v t . By the convexity of f t (· ) , f t ( 0 ) = 0, we have f t ( v t ) ≤ α v t , ∀ v t ≤ ˆ v t . T hen д t ( v t ) ≥ ˜ д t ( v t ) , ∀ v t ≤ ˆ v t Suppose ˜ v t is the smaller sol ution satisfying ˜ д t ( ˜ v t ) = k , i.e ., ˜ v t = p ( t ) − p p 2 ( t ) − 4 α k 2 α = 2 k p ( t )  1 + q 1 − 4 α k p 2 ( t )  . By observing k π ≤ д t ( ˆ v t ) = ˜ д t ( ˆ v t ) ≤ p 2 ( t ) 4 α , we have 4 α k p 2 t ≤ 1 π (note that this also implies the existence of ˜ v t ). W e then easily conclude ˜ v t ≤ 2 k p ( t )  1 + q 1 − 1 π  . W e claim ¯ v t ≤ ˜ v t . If ˜ v t > ˆ v t , we have ¯ v t ≤ ˆ v t < ˜ v t ; Otherwise, ˜ v t ≤ ˆ v t , we have k = д t ( ¯ v t ) = ˜ д t ( ˜ v t ) ≤ д t ( ˜ v t ) . Following д t ( v t ) is increasing in [ 0 , ˆ v t ] , we conclude ¯ v t ≤ ˜ v t . Finally , we conclude ¯ v t ≤ 2 k p ( t )  1 + q 1 − 1 π  = 2  1 + q 1 − 1 π  д t ( ¯ v t ) p ( t ) .  A.11 Proof of Lemma 15 Proof. From Lemma 14, we have Φ ∆ ( π ) = max σ [ 1: T ] T Õ t = 1 v t ≤ 2  1 + q 1 − 1 π  T Õ t = 1 д t ( ¯ v t ) p ( t ) . By Lemma 11, we know that T Õ t = 1 д t ( ¯ v t ) p ( t ) ≤ ∆ π ( 1 + ln θ ) . (26) Then we can bo und Φ ∆ ( π ) as Φ ∆ ( π ) ≤ 2  1 + q 1 − 1 π  T Õ t = 1 д t ( ¯ v t ) p ( t ) ≤ 2 ∆ π  1 + q 1 − 1 π  ( 1 + ln θ ) . This completes our proo f. 