Transmission with Energy Harvesting Nodes in Fading Wireless Channels: Optimal Policies

T ransmission with Ener gy Harv esting No des in F ading W irele s s Chann els: Optimal Policie s Omur Ozel, Kaya T utuncuoglu, Jing Y ang, Sennur Ulukus and A ylin Y ener Abstract W ireless systems comprised of rechargeable nodes hav e a significantly p rolonged lifetime and are sustainable. A distinct char acteristic of th ese systems is the fact tha t the nodes can h arvest energy throug hout the duration in which comm unication takes place. As such, transmission po licies of the nodes need to adap t to these har vested energy arri vals. I n this pap er , we consider optimization o f point- to-poin t data transmission with an en ergy h arvesting transmitter which has a limited b attery c apacity , commun icating in a wireless fading channel. W e consider two o bjectives : maximizing the throug hput by a deadline, and minimizing the tra nsmission c ompletion time of the co mmunicatio n session . W e optimize th ese objectives by con trolling the time sequence of transmit p owers subject to energy storage capacity and causality con straints. W e, first, stud y optimal offline policies. W e introdu ce a d ir ection al water -filling alg orithm which p rovides a simple an d concise inter pretation of the necessary optim ality condition s. W e show the o ptimality of an adaptive directional water-filling algorithm for the throu ghput maximization pr oblem. W e solve th e transmission co mpletion time m inimization pro blem by utilizing its eq uiv alence to its through put maxim ization counterp art. Next, we consider onlin e p olicies. W e u se stochastic dynamic prog ramming to solve for the op timal o nline policy that maximizes th e average number o f b its delivered by a deadline under stoch astic fading an d energy ar riv al p rocesses with causal channel state feedba ck. W e also propose near-optimal policies with redu ced comp lexity , and nu merically study their perfor mances along with the perfo rmances of the offline an d onlin e o ptimal policies under various different configuratio ns. Index T erms Energy ha rvesting, re chargeable wireless n etworks, throughp ut maximizatio n, transmission co mple- tion time minim ization, directio nal water -filling, d ynamic pr ogramm ing. Omur Ozel and Sennur Ulukus are with the Department of Electrical and Computer Engineering, University of Maryland, College Park, MD 20742, USA. Kaya Tutun cuoglu and A ylin Y ener are with the Department of Electrical Engineering, Pennsylv ania State Univ ersity , Univ ersity Park, P A 16802, USA. Jing Y ang was with the Department of Electrical and Computer Engineering, Uni versity of Maryland, College Park , MD 20742 , USA . She is n o w with the Dep artment of Electrical and Computer Engineering, Uni versity of W isconsin-Madison , WI 53706, USA. This work was supp orted by NSF Grants CNS 09-64632, CNS 09-64364 and presented in part at the Conference on Information Sciences and Systems (CISS ), Baltimore, MD, March 2011 and at IEEE International Conference on Computer Communications (INFOCOM), Shangai, China, April 2011. I . I N T R O D U C T I O N This paper considers wireless com munication usi ng energy harvesting transmit ters. In such a scenario, incremental e ner gy is ha rvested by the transmitter during the course of data transmission from random ener gy sources. As such, ener gy becomes a va ilable for packet transmission at random times and in random amounts. In addition, the wireless communication channel fl uctuates randomly due to fading. These togeth er lead to a need for designing new transmission strategies that ca n best take adv antage of and adapt to the random ener gy arriv als as well as channel var iations i n time. The simplest syst em model for which thi s setting leads to new design insigh ts is a wireless lin k with a rechar geable transm itter , whi ch we cons ider here. The incom ing ener g y can be sto red in the battery of the rechar geable transm itter for future use. Howe ver , this battery has finite storage capacity and the transmiss ion pol icy needs to guarantee that there is suffi cient battery space for each energy arriv al, otherwise incomi ng energy cannot be sav ed and will be wasted. In th is setting, we find optimal offline and on line transm ission schemes that adapt the i nstantaneous transmit p owe r to the variations in the energy and fade leve ls. There has been recent research ef fort on und erstanding dat a transm ission wit h an energy har - vesting transmi tter that has a rechar g eable battery [1]–[7]. In [1], dat a transmissio n with energy harvesting senso rs is considered, and the optimal online policy for controlling adm issions into the data buf fer is deriv ed u sing a dynam ic programmi ng framework. In [2], ener gy m anagement policies which stabi lize the data queue are proposed for single-user communi cation and under a linear approx imation, some delay optimali ty properties are derived. In [3], the optimalit y of a var iant of the back pressure algorithm using energy q ueues is shown. In [4], throu ghput optimal energy allo cation is studied for energy harvesting systems in a time con strained slott ed setting. In [5], [6], minim ization of t he transmis sion completion tim e is consi dered in an energy harvesting sys tem and the opti mal solution is obtained using a geometric framew o rk similar to the calculus approach presented in [8]. In [7], energy harvesting transm itters with batteries of finite energy storage capacity are considered and the problem of throughp ut maximization by a deadline i s solved in a stat ic channel. An earlier line of research con sidered the problem of energy management in communi cations 1 satellites [9], [10]. In [9], various energy allocatio n probl ems in solar po wered comm unication satellites are solved using dynamic programming. In [10], optimal ener gy al location to a fixed number of time slot s is deriv ed under time-varying channel g ains and with offl ine and on line knowledge of the channel state at the transmitt er . Another related line of research consi dered ener g y minimal transm ission problems with deadline constraints [8], [11]–[13]. Our work pro- vides opti mal transmis sion policies to maximize the throughput and minimize t he transmissi on completion tim e, under channel fluctuations and energy variations, in a cont inuous ti me model, generalizing these related works, [4]–[8], [10]–[13], from various di f ferent perspectives. In particul ar , we consider two related opti mization problems. The first problem i s the maxi- mization of the number of bits (or through put) transmi tted by a deadline T . Th e second problem is the minimi zation of the time (or delay) by which the transmissi on of B bits is completed. W e solve the first problem under d eterministic (offline) and stochast ic (online) sett ings, and we solve the second problem in the det erministic setting. W e start the analysis by consi dering th e first prob lem in a stati c channel under offline knowledge. The solutio n call s for a new algorit hm, termed dir ectio nal water-filling . T aking into account the causality constraints on the ener gy usage, i.e., the energy can b e saved and used in the futu re, the algori thm allows energy flow on ly to the right. In the algorit hmic implem entation of t he s olution, we utili ze right permeab le t aps at each ener g y arriv al point. This s olution serves as a building block for the fading case. Specific ally , we sh o w that a directional water -filling algorithm that adapts to b oth energy arriv als and channel fade lev els is o ptimal. Next, we consider the second problem, i.e., th e minimi zation of t he time by which transmi ssion of B bits is com pleted. W e use th e sol ution o f the first problem to sol ve this second probl em. This is accomplished by mapping the first problem to t he second problem by means of the ma ximum departur e curve . This compl etes the identification of the optim al offl i ne policies i n the fading channel. Next, we set out to find online policies. W e address onli ne scheduling for maximu m through- put by the deadline T in a sett ing where fading lev el changes and ener gy arrives as random processes in time. Assuming statisti cal kno wledge and causal in formation of the ener gy and fading var iations, we solve for the opt imal o nline power poli cy by using cont inuous time stochastic dyn amic programmin g [13 ], [14]. T o reduce the comp lexity required by the d ynamic 2 programming solu tion, we propose s imple onl ine algorithms that perform near-optimal. Finally , we p rovide a thorough num erical study of th e proposed algorithms under var ious system settin gs. I I . S Y S T E M M O D E L W e consider a si ngle-user fading channel with additiv e Gaus sian noi se and causal channel state information (CSI) feedback as shown i n Fig. 1. Th e t ransmitter has t wo queues, the data queue where data p ackets are s tored, and an energy queue where the arriving (harvested) energy is s tored. The energy queue, i.e., the battery , can st ore at m ost E max units of ener gy , whi ch is used o nly for transmissi on, i.e., energy required for processing is not cons idered. The recei ved signal y is given by y = √ hx + n , where h is the (squared) fading, x is t he channel in put, and n i s a Gaussian rando m no ise wit h zero-mean and unit -v ariance. Whenev er an input signal x is t ransmitted with p owe r p in an epoch o f duration L , L 2 log (1 + hp ) bi ts o f data is served out from t he b acklog with t he cost of Lp u nits of energy depletion from the energy queue. Th is fol lows from the Gaussian channel capacity formula. The bandwidth is sufficiently wide so that L can t ake s mall values and we approximate the s lotted system to a continuous time syst em. Hence, we say that if at time t the transmit power of the signal is x 2 ( t ) = p ( t ) , the instantaneou s rate r ( t ) i n bi ts per channel u se is r ( t ) = 1 2 log (1 + h ( t ) p ( t )) (1) Follo w ing a m odel similar to [12], we assume that the fading l e vel h and energy arri vals are stochastic processes in time that are marked by Poisson counting processes with rates λ h and λ e , respective ly . Therefore, changes i n fading level and energy arri vals occur in countable time ins tants, which are indexed respectiv ely as t f 1 , t f 2 , . . . , t f n , . . . and t e 1 , t e 2 , . . . , t e n , . . . wi th the con vention that t e 1 = t f 1 = 0 . By t he Poisson property , the inter-occ urrence ti mes t f i − t f i − 1 and t e j − t e j − 1 are exponentially distributed with means 1 / λ f and 1 /λ e , respecti vely . The fading lev el in [0 , t f 1 ) is h 1 , in [ t f 1 , t f 2 ) is h 2 , and so on. Simil arly , E i units of ener gy arri ves at time t e i , and E 0 units o f energy is av ail able at time 0 . Hence { ( t e i , E i ) } ∞ i =0 and { ( t f i , h i ) } ∞ i =1 completely define the events that take place during the course o f data transm ission. This model is shown in Fig. 2. The incoming ener gy is first buf fered in the battery b efore it is us ed in data transmiss ion, and 3 energy queue data queue causal CSI feedback √ h N E max Tx Rx E in Fig. 1. Additiv e Gaussian fading channel with an energ y harvesting transmitter and causal channel st ate information (CSI) feedback. the transm itter is allowed to use the battery energy onl y . Accordingly , we assume E i ≤ E max for all i as o therwise excess ener gy cannot be accommodated in th e b attery anyway . In t he sequel, we wi ll refer to a change in the channel fading le vel or i n th e energy level as an event and the t ime interval between two consecutive events as an epoch . Mo re precisely , epoch i is defined as th e time interval [ t i , t i +1 ) where t i and t i +1 are the times at which s uccessiv e e vents happen and th e lengt h of the epoch is L i = t i +1 − t i . Natu rally , energy arriv al informati on is causally av ailabl e to the trans mitter . Moreover , by virtue of t he causal feedback link, perfect information of the channel fade lev el is av ailable to the transmi tter . Therefore, at time t all { E i } and { h j } such that t e i < t and t f j < t are known perfectly b y th e t ransmitter . A power m anagement policy is denoted as p ( t ) for t ∈ [0 , T ] . T here are two constraints on p ( t ) , due to energy arri vals at random times and also d ue to finite battery storage capacity . Since ener g y t hat has n ot arrived y et cannot be used at the current t ime, there is a causality constraint on t he power management policy as: Z t e i 0 p ( u ) du ≤ i − 1 X j = 0 E j , ∀ i (2) where the limit of the int egral t e i should be i nterpreted as t i e − ǫ , for small enou gh ǫ . Mo reov er , due to the finite battery storage capacity , we need to m ake sure that energy level in the b attery nev er exceeds E max . Since ener gy arriv es at certain time points, it is sufficient to ensure that the energy le vel in the b attery nev er exceeds E max at the tim es of ener g y arriv als. Let d ( t ) = 4 0 T epoch 1 epoch 7 epoch 10 epoch 15 fading levels E 0 E 1 E 2 E 3 E 4 E 5 Fig. 2. The system model and epochs under channe l fading. max { t e i : t e i ≤ t } . Then, d ( t ) X j = 0 E j − Z t 0 p ( u ) du ≤ E max , ∀ t ∈ [0 , T ] (3) W e emphasize that our system m odel is cont inuous rather than slotted. In slotted m odels, e.g., [3], [4], [9], the energy input-output relations hip is written for an entire sl ot. Such models allow ener g ies larger than E max to enter the battery and b e used for transmissio n in a g iv en single slot. Our contin uous syst em model prohibi ts such occurrences. Our ult imate g oal is to dev elop an online algorithm th at determines th e transmi t power as a function of t ime u sing the causal knowledge of the system, e.g., the ins tantaneous ener gy state and fading CSI. W e will start our development by considering the optim al offline policy . I I I . M A X I M I Z I N G T H RO U G H P U T I N A S T A T I C C H A N N E L In this section, we consider maximizin g the number of bits deliv ered by a deadline T , in a non- fading channel with of fline knowledge of energy arriv als w hich o ccur at times { t 1 , t 2 , . . . , t N } in amounts { E 1 , E 2 , . . . , E N } . The epoch l engths are L i = t i − t i − 1 for i = 1 , . . . , N wit h t 0 = 0 , and L N +1 = T − t N . There are a total of N + 1 epochs. The opti mization is subject to causality constraints on the harvested energy , and t he finite storage constraint on the rechargeable battery . This problem was solved in [7] using a geometric framework. Here, we provide the form ulation for completeness and provide an alternative algorithm ic solut ion which will serve as the building block for the solu tion for th e fading channel presented in the next section. First, we note t hat the transm it po wer must be kept const ant in each epoch [5]–[7], due to th e conca vity of rate in p owe r . Let us denote th e p owe r in epoch i by p i . The causality constraints 5 in (2) reduce to the fol lowing constraints on p i , ℓ X i =1 L i p i ≤ ℓ − 1 X i =0 E i , ℓ = 1 , . . . , N + 1 (4) Moreover , since the ener gy le vel in the b attery is the highest at ins tants when ener gy arri ves, the battery capacity constraint s in (3) reduce t o a countable number of const raints, as foll ows ℓ X i =0 E i − ℓ X i =1 L i p i ≤ E max , ℓ = 1 , . . . , N (5) Note that since E 0 > 0 , there is no incentive to make p i = 0 for any i . Hence, p i > 0 is necessary for optimali ty . The opt imization problem is: max p i ≥ 0 N +1 X i =1 L i 2 log (1 + p i ) (6) s.t. ℓ X i =1 L i p i ≤ ℓ − 1 X i =0 E i , ℓ = 1 , . . . , N + 1 (7) ℓ X i =0 E i − ℓ X i =1 L i p i ≤ E max , ℓ = 1 , . . . , N (8) W e note t hat the const raint in (7) must be satisfied w ith equality for ℓ = N + 1 , oth erwise, we can always i ncrease some p i without conflicting any other const raints, increasing the resulting number of bits transmi tted. Note t hat the objecti ve function in (6) is conca ve i n the vector of powers si nce it is a sum of log functions, whi ch are concav e t hemselves. In addition, the const raint set is con vex as it is composed of linear constraints. Hence, the above op timization problem is a con vex optim ization problem, and h as a uni que m aximizer . W e define the following Lagrangian function [15] for any λ i ≥ 0 and µ i ≥ 0 , L = N +1 X i =1 L i 2 log (1 + p i ) − N +1 X j = 1 λ j j X i =1 L i p i − j − 1 X i =0 E i ! − N X j = 1 µ j j X i =0 E i − j X i =1 L i p i − E max ! (9) Lagrange multipli ers { λ i } are associated wit h constraints in (7) and { µ i } are associated wit h 6 (8). Addition al complim entary slackness condition s are as fol lows, λ j j X i =1 L i p i − j − 1 X i =0 E i ! = 0 , j = 1 , . . . , N (10) µ j j X i =0 E i − j X i =1 L i p i − E max ! = 0 , j = 1 , . . . , N (11) In (1 0), j = N + 1 is not included s ince this constraint is i n fact sati sfied with equality , because otherwise th e objective function can be increased by increasing so me p i . Note also that as p i > 0 , we did not include any slackness conditio ns for p i . W e apply the KKT optimality conditions to t his Lagrangian to obtain the op timal power l e vels p ∗ i in terms of th e Lagrange mul tipliers as, p ∗ i = 1  P N +1 j = i λ j − P N j = i µ j  − 1 , i = 1 , . . . , N (12) and p ∗ N +1 = 1 λ N +1 − 1 . Note that p ∗ i that satisfy P N +1 i =1 L i p ∗ i = P N i =0 E i is unique. Based on the expression for p ∗ i in terms o f the Lagrange multipliers in (12), we have the following observation on the structure of the op timal power allocation scheme. Theor em 1 When E max = ∞ , the opt imal power lev els is a monotonically increasing sequence: p ∗ i +1 ≥ p ∗ i . M oreov er , if for so me ℓ , P ℓ i =1 L i p ∗ i < P ℓ − 1 i =0 E i , then p ∗ ℓ = p ∗ ℓ +1 . Pr oof: Since E max = ∞ , constraints in (8) are satisfied without equali ty and µ i = 0 for all i by slackness conditions in (11). From (12), since λ i ≥ 0 , optimum p ∗ i are monotonically in creasing: p ∗ i +1 ≥ p ∗ i . Moreover , if for some ℓ , P ℓ i =1 L i p ∗ i < P ℓ − 1 i =0 E i , t hen λ ℓ = 0 , which means p ∗ ℓ = p ∗ ℓ +1 .  The monotonicity in Theorem 1 is a resul t of the fact that energy may be spread from t he current tim e to the future for optimal operation. Whenever a constraint in (7) is not satisfied with equality , it means that some energy is av ailable for use but is n ot used in the current epoch and is transferred to future epochs. Hence, th e o ptimal power al location is s uch that, if some ener g y is transferred to future epo chs, then the power level must remain the same. H o we ver , if the optim al power level changes from epoch i to i + 1 , then th is change should b e in the form of an i ncrease and no energy is transferred for future use. That is, the corresponding constraint 7 in (7) is satis fied with equalit y . If E max is finit e, then its effect on the op timal power allo cation is observed through µ i in (12). In particular , i f the constraints in (8) are satisfied witho ut equality , then opt imal p ∗ i are still m onotonically increasing since µ i = 0 . Ho we ver , as E i ≤ E max for all i , t he constraint with the same index in (7) is satisfied with out equality whenever a constraint in (8) is satisfied with equality . T herefore, a non-zero µ i and a zero λ i appear in p ∗ i in (12). This im plies that the monotonici ty of p ∗ i may no l onger hold. E max constraint restricts power lev els to take the same value in adjacent epochs as it constrains the ener g y that can be transferred from current epoch to the future epochs. Indeed, from constraint s in (8), the energy that can be t ransferred from current, say the i th, or previous epochs, to future epochs is E max − E i . Hence, the power levels are equalized only to t he extent that E max constraint allows. A. Dir ectiona l W ater -F il ling Algorit hm W e interpret the o bserved properties of the optim al power all ocation scheme as a dir ection al water- filling scheme. W e note that if E units of water (energy) is filled i nto a rectangle of bot tom size L , th en t he water level is E L . Another key ingredient of the directional water -filling alg orithm is the concept of a ri ght permeable tap, which permit s transfer of water (energy) only from left to right. Consider the two epoch system. As sume E max is s uf ficiently l ar ge. If E 0 L 1 > E 1 L 2 , then some ener g y is transferred from epoch 1 to epoch 2 so t hat t he lev els are equalized. This is sh o wn in the top figure i n Fig. 3. Howe ver , if E 0 L 1 < E 1 L 2 , no ener gy can flow from rig ht to left. This is due to t he causality of ener gy usage, i.e., energy cannot be used before it is harvested. Therefore, as shown in the middle figure in Fig. 3, the water le vels are not equalized. W e impl ement t his using right permeable taps, which let water (energy) flow only from left to right. W e not e t hat the fini te E max case can be incorporated i nto the energy-water analogy as a constraint on the amount of energy t hat can be transferred from the past to the future. If the equalizing water level requires more than E max − E i amount of ener g y to be transferred, t hen only E max − E i can be t ransferred. Because, otherwis e, the ener gy level in the next epoch exceeds E max causing overflo w of energy , which results i n ineffic iencies. More s pecifically , when the right p ermeable tap in between the two epochs o f th e example in bot tom figure in Fig. 3 is 8 0 T ON 0 T OFF L 1 L 2 E 0 + E 1 L 1 + L 2 L 1 L 2 E 1 L 2 E 0 L 1 E 1 E 0 0 T OFF 0 T ON L 1 L 2 E 0 L 1 E 1 L 2 E 1 E 0 L 1 L 2 E 0 L 1 E 1 L 2 0 T ON 0 T OFF E max L 2 L 1 L 2 E max E max L 1 L 2 E 1 L 2 E 0 L 1 E max E max E 1 E 0 Fig. 3. Directional water-filling wit h right permeable taps in a two-epoch setti ng. turned o n, only E max − E 1 amount of energy transfer i s allowed from epoch 1 to epoch 2 . I V . M A X I M I Z I N G T H R O U G H P U T I N A F A D I N G C H A N N E L W e now solve for the offline poli cy for the fading channel util izing the ins ights obtain ed in the pre vious s ection. The channel state changes M times and energy arri ves N times i n t he duration [0 , T ) . Hence, we ha ve M + N + 1 epo chs. Our goal is again to m aximize t he numb er of bits transmitted by the deadline T . Simi lar to the non-fading case, the optimal power management strategy is such that the transmit power is constant i n each event epoch. Therefore, let us again denote t he transmit power in epoch i by p i , for i = 1 , . . . , M + N + 1 . W e define E in ( i ) as the ener g y which arri ves in epoch i . Hence, E in ( i ) = E j for som e j if event i is an energy arriv al and E in ( i ) = 0 if ev ent i is a fade l e vel change. Also, E in (1) = E 0 . Simi lar to the non-fading case, we ha ve causality constraints due to energy arriv als and an E max constraint due to finite 9 battery size. Hence, the opt imization pro blem i n th is fading case becomes: max p i ≥ 0 M + N +1 X i =1 L i 2 log (1 + h i p i ) (13) s.t. ℓ X i =1 L i p i ≤ ℓ X i =1 E in ( i ) , ∀ ℓ (14) ℓ X i =1 E in ( i ) − ℓ X i =1 L i p i ≤ E max , ∀ ℓ (15) Note that, as in the no n-fading case, the constrain t in (14) for ℓ = M + N + 1 must b e sati sfied with equality , since otherwis e, we can always increase one of p i to increase the throughp ut. As i n the non-fading case, the objective funct ion in (13) i s concave and the constraints are con vex. The optimi zation problem has a uni que optimal soluti on. W e define the Lagrangian for any λ i , µ i and η i as, L = M + N +1 X i =1 L i 2 log (1 + h i p i ) − M + N +1 X j = 1 λ j j X i =1 L i p i − j X i =1 E in ( i ) ! − M + N +1 X j = 1 µ j j X i =1 E in ( i ) − j X i =1 L i p i − E max ! + M + N +1 X i =1 η i p i (16) Note that we have not emp loyed the Lagrange multipliers { η i } in the n on-fading case, sin ce in that case, we need to have all p i > 0 . Howe ver , in the fading case, s ome of the optimal powers can be zero depend ing on the channel fading state. Associated complimentary slackness conditions are, λ j j X i =1 L i p i − j X i =1 E in ( i ) ! = 0 , ∀ j (17) µ j j X i =1 E in ( i ) − j X i =1 L i p i − E max ! = 0 , ∀ j (18) η j p j = 0 , ∀ j (19) It follows that the opti mal powers are given by p ∗ i =  ν i − 1 h i  + (20) 10 where the water lev el in epoch i , ν i , is ν i = 1 P M + N +1 j = i λ j − P M + N +1 j = i µ j (21) W e have the following observation for the fading case. Theor em 2 When E max = ∞ , for any epoch i , the o ptimum water level ν i is m onotonically increasing: ν i +1 ≥ ν i . Moreover , if some energy is transferred from epoch i to i + 1 , th en ν i = ν i +1 . Pr oof: E max = ∞ ass umption results in µ i = 0 for all i . From (21), and since λ i ≥ 0 , we hav e ν i +1 ≥ ν i . If energy is transferred from the i th epoch to the i + 1 st epoch, then the i th constraint in (14) is satisfied withou t equality . Thi s implies, by slackness conditions in (17), that for thos e i , we hav e λ i = 0 . Hence, by (21), ν i = ν i +1 . In particular , ν i = ν j for all epochs i and j that are in between two consecutive ener gy arriv als as there is no wall between these epochs and injected ener g y freely sp reads into these epochs.  As in t he non-fading case, the effect of finite E max is observed via the Lagrange multipl iers µ i . In particular , wheneve r E max constraint is s atisfied with equality , the monot onicity of the water lev el no longer ho lds. E max constrains the amount of ener gy that can be transferred from o ne epoch to th e next. Specifically , the t ransferred ener gy cannot be larger than E max − E in ( i ) . Note that th is constraint is trivially satisfied for those epochs with E in ( i ) = 0 because E in ( i ) < E max and hence the water lev el in between t wo ener gy arriv als must be equalized. Howev er , the next water level may be higher or lower d epending on t he new arri v ing energy amount. A. Dir ectiona l W ater -F il ling Algorit hm The directional water -filling algorithm in the fading channel requires walls at the point s of ener g y arriv al, with right permeable water taps in each wall wh ich allows at most E max amount of water to flow . No walls are required to separate the epochs due to changes in the fading level. The water levels when each right permeable t ap is turned on wi ll be found by t he directional water -filling algorithm. Optimal power allocation p ∗ i is th en calculated by pluggi ng the result ing water le vels into (20). An e xample run of the algorithm is sho wn in Fig. 4, for a case of 12 11 0 water level fade level 0 ON ON ON OFF OFF OFF T L 1 L 2 L 3 L 4 L 5 L 6 L 7 L 8 L 9 L 10 L 11 L 12 ( 1 h i ) T L 1 L 2 L 3 L 4 L 5 L 6 L 7 L 8 L 9 L 10 L 11 L 12 E max E max E max E max E max E max E max E max E 3 E 0 E 2 E 1 Fig. 4. Directional water-filling wit h right permeable taps in a fading channel. epochs. Three energy arriv als occur during the course o f t he t ransmission, in addition to the ener g y a vailable at time t = 0 . W e observe that the ener gy le vel equalizes in epochs 2, 4, 5, while no power is transmi tted in epochs 1 and 3, since the channel gains in these epochs are too low (i.e., 1 h i too high). The energy arriving at the b eginning of epoch 6 cannot flow left due to causality constraints, which are enforced by right permeable taps, whi ch allow ener gy flo w only to the right. W e observe that the energy equal izes between epochs 8 throu gh 12, howe ver , the excess ener gy in epochs 6 and 7 cannot flo w right, du e to the E max constraint enforced by the right permeable tap between epochs 7 and 8. V . T R A N S M I S S I O N C O M P L E T I O N T I M E M I N I M I Z A T I O N I N F A D I N G C H A N N E L In contrast to the infinite backlog assumpti on of the previous sections , we now assum e that the transmitter has B bits to be communi cated to the recei ver in the ener gy harve sting and fading channel setting. Our objective now is to minimize the time necessary t o transm it t hese B bits. This probl em is called the transmissio n completi on time mi nimization problem. In [5], [6], this 12 problem i s formulated and s olved for an ener gy harvesting sy stem in a non -fading en vironment. In [7], the problem is solved when there i s an E max constraint on the energy buffe r (battery) by identifying its connection to its throughput optimization counterpart. Here, our goal is to address this problem in a fading channel, by using the directional water -filli ng approach we hav e deve loped so far . The transm ission completion time minim ization problem can b e stated as, min T (22) s.t. N X i =1 L i 2 log (1 + h i p i ) = B (23) ℓ X i =1 L i p i ≤ ℓ X i =1 E in ( i ) , ℓ = 1 , . . . , N (24) ℓ X i =1 E in ( i ) − ℓ X i =1 L i p i ≤ E max , ℓ = 1 , . . . , N (25) where N , N ( T ) is the num ber of epochs in the interval [0 , T ] . The solution wi ll be a generalization of the result s in [5]–[7] for the fading case. T o this end, we in troduce the maximum departure curve. T his maximum departure curve function will m ap the t ransmission completion time mi nimization problem of this s ection t o the through put maxim ization problem of the previous sections. A. Maximum Departur e Curve Giv en a deadline T , define the maxi mum departure curve D ( T ) for a g iv en sequence of energy arri va ls and channel fading stat es as, D ( T ) = max N ( T ) X i =1 L i 2 log (1 + h i p i ) (26) where N ( T ) is the num ber of epochs i n the in terv al [0 , T ] . The maximizatio n in (26 ) is subj ect to the energy causalit y and m aximum battery storage capacity constraint s i n (24) and (25). The maximum departure function D ( T ) represents the maxim um num ber of bi ts t hat can be served out of the backlog by the deadline T given the energy arri val and fading sequences. This is exactly the solution o f the problem st udied in th e pre vious sections. Some characteristics of the 13 maximum departure curve are s tated in the foll o wing lemm a. Lemma 1 Th e maximum departure curve D ( T ) is a m onotonically increasing and continuous function of T . D ( T ) is not d if ferentiable at { t e i } and { t f i } . Pr oof: The mon otonicity fol lows because as the deadline is increased, we can transmit at least as many bits as we could with the s maller deadline. The con tinuity follows by observing th at, if no new energy arrives or fading st ate changes, there is n o reason to have a discont inuity . When new ener gy arriv es, s ince the number of bits that can be t ransmitted with a finite amount of ener g y is finit e, the number of bi ts transmitted will n ot ha ve any ju mps. Sim ilarly , if th e fading lev el changes, due to the continuity of the log function, D ( T ) wil l be continuous. For the non-differ entiable poi nts, assum e that at t = t e i , an energy in the amount of E i arri ves. There exists a s mall enough increment from t e i that the water lev el on the right is higher than the water lev el on the l eft. The right permeable taps wil l not allow this water to flo w to left. Then, the D ( T ) will have the following form: D ( t e i + ∆) = D ( t e i ) + ∆ 2 log  1 + E i h ∆  (27) Thus, the right deriv ative of D ( T ) at t = t e i , becomes arbitrarily large. Hence, D ( T ) is not diffe rentiable at t e i . At t = t f i , th e fade lev el changes from h i to h i +1 . As t is increased, water lev el decreases unless new energy arri ves. The change in the water level i s proportional to 1 h i +1 for t > t f i and is proportional to 1 h i for t < t f i . Hence, at t = t f i , D ( T ) is n ot di f ferentiable.  The conti nuity and monotonicity of D ( T ) implies that the in verse function of D ( T ) exists, and that for a closed interval [ a, b ] , D − 1 ([ a, b ]) is als o a closed int erv al. Since D ( T ) is obtain ed by the directional water -filling algorith m, th e deri vati ve of D ( T ) has the i nterpretation of the rate of ener gy transfer from past int o the future at time T , i.e., it is the measure of t he tendency of water to flow right. T he non-di f ferentiabi lities at energy arri val and fading change points are compatible with this interpretation. W e can visualize the result of Lem ma 1 by cons idering a few simpl e examples. As the simplest example, consider th e n on-fading channel ( h = 1 ) with E 0 units of ener gy a vailable at th e transmitter (i .e., no ener gy arriv als). Then, t he optimal transmissi on scheme is a constant 14 transmit p owe r scheme, and hence, we have, D ( T ) = T 2 log  1 + E 0 T  (28) It i s clear that t his is a conti nuous, monoto nically increasing funct ion, whose deri vati ve at T = 0 (at the tim e of energy arri val) is un bounded. Next, we consider a two epoch case where E 1 arri ves at T 1 and fading l e vel is constant (and also h = 1 ). W e assume E 0 and E 1 are both s maller th an E max and E 0 + E 1 > E max . After some algebra, D ( T ) can shown t o be e xpressed as, D ( t ) =                t 2 log  1 + E 0 t  , 0 < t < T 1 T 1 2 log  1 + E 0 T 1  + t − T 1 2 log  1 + E 1 t − T 1  , T 1 ≤ t ≤ T 2 t 2 log  1 + E 0 + E 1 t  , T 2 < t < T 3 T 3 2 log  1 + E 0 + E 1 − E max T 3  + t − T 3 2 log  1 + E max t − T 3  , T 3 < t < ∞ (29) where T 2 = E 1 T 1 E 0 + T 1 , T 3 = T 1 ( E 0 + E 1 ) E 0 + E 1 − E max . In this E max constrained case, the asymp tote of D ( T ) as T → ∞ is st rictly smaller than that in E max = ∞ case. In the mos t general case where we have multi ple energy arriv als and channel state changes, these basi c properties will follow . An example case i s sh o wn in Fig. 5. Note that there m ay be discontinui ties in D ′ ( T ) due to o ther reasons t han fading lev el changes and energy arriv als, such as the E max constraint. B. Solutio n of the T ransmission Completion T ime Minimiz a tion Pr oblem in a F ad i ng Channel W e no w solve the t ransmission com pletion time m inimization problem stated in (22)-(25). Minimizati on of the time to complete the transmission of B bi ts av ailable at the transmi tter is closely related with t he maximization of the numb er of bits that can be sent by a deadlin e. In fact, if t he maxim um number of bits that can be sent by T is less than B , then it is not possibl e to complete the t ransmission of B bits by T . As we state formal ly below , if T ∗ is the minim al time to com plete the transmission of B bits, then necessarily B = D ( T ∗ ) . Thi s argument provides a characterization for T ∗ in terms of the maxim um departure curve, as stated in the following theorem. 15 D(T) T E 0 E 1 E 2 E 3 Fig. 5. The general form of the maximum departure curve. Theor em 3 The minim um transmissi on completion ti me T ∗ to transmit B bit s i s T ∗ = min { t ∈ M B } where M B = { t : B = D ( t ) } . Pr oof: For t su ch t hat D ( t ) < B , T ∗ > t since the maximum number of bits that can be served by t is D ( t ) and i t is less th an B . Hence, B bits canno t be compl eted by t . Con versely , for t such t hat D ( t ) > B , T ∗ < t b ecause B bits can be completed by t . Hence, D ( T ∗ ) = B is a necessary condi tion. As D ( T ) is continuou s, the set { t : B = D ( t ) } is a closed set. Hence, min { t : B = D ( t ) } exists and is un ique. By t he definition of T ∗ , we ha ve T ∗ = min { t : B = D ( t ) } .  V I . O N L I N E T R A N S M I S S I O N P O L I C I E S In this section, we wil l st udy scheduling in the given setting with onl ine, i.e., causal, infor- mation of the events. In particular , we consider the m aximization of th e number of bits sent by deadline T given only causal informati on of the energy arri vals and channel fade levels at the transmitter side as in Fig. 1 . W e assum e that th e energy arri val is a compoun d Poisson process with a density function f e . Hence, N e is a Poisson random variable wi th m ean λ e T . The channel fade lev el is a stochasti c 16 process marked with a Poisson process of rate λ f . Thus, N f is Poi sson with m ean λ f T . The channel takes ind ependent values with p robability density f h at each marked tim e and remai ns constant in between two marked po ints. A. Optimal Online P ol icy The states of the sy stem are fade lev el h and batt ery energy e . An online pol icy i s denoted as g ( e, h, t ) which denotes th e transmit power decided by the t ransmitter at ti me t given the states e and h . W e call a policy admissi ble i f g is nonnegati ve, g (0 , h, t ) = 0 for all h and t ∈ [0 , T ] and e ( T ) = 0 . That is , we i mpose an i nfinite cost if the remainin g energy in the b attery is non -zero after the deadline. Hence, admi ssible policies guarantee that no transmiss ion can occur if the battery ener gy is zero and ener gy left in the battery at th e t ime of the deadline is zero so that resources are used ful ly by the deadlin e. T he throughput J g ( e, h, t ) is the expected number of bits sent by the tim e t und er t he policy g J g ( e, h, t ) = E  Z t 0 1 2 log (1 + h ( τ ) g ( e, h, τ )) dτ  (30) Then, the value function is t he supremu m over all admi ssible poli cies g J ( e, h, t ) = sup g J g (31) Therefore, the optimal online pol icy g ∗ ( e, h, t ) is such that J ( e, h, t = 0) = J g ∗ , i.e., it solves the following problem max g E  Z T 0 1 2 log (1 + h ( τ ) g ( e, h, τ )) d τ  (32) In order to solve (32), we first cons ider δ -skeleton o f the random processes [13]. For suf ficiently small δ , we quantize t he time by δ and ha ve the following. max g E  Z T 0 1 2 log (1 + h ( τ ) g ( e, h, τ )) dτ  = max g ( e,h, 0)  δ 2 log (1 + h (0) g ( e, h, 0)) + J ( e − δ g ( e, h, 0) , h, δ )  (33) Then, we can recursiv el y solve (33 ) to obtai n g ∗ ( e, h, t = k δ ) for k = 1 , 2 , . . . , ⌊ T δ ⌋ . Thi s procedure is the dynamic programm ing solution for continuous time and the outcom e i s the 17 optimal online poli cy [13], [14]. After solv ing for g ∗ ( e, h, t ) , the transmit ter records thi s function as a lo ok-up t able and at each time t , it recei ves feedback h ( t ) , s enses the battery energy e ( t ) and transmits with power g ∗ ( e ( t ) , h ( t ) , t ) . B. Other Online P o l icies Due to the curse of dimensionali ty inh erent in the dynami c programming s olution, it is natural to forgo performance in lieu of less complex onl ine policies. In th is subsection, we propose sev eral subop timal transmiss ion pol icies that can s omewhat mimi c th e offline opti mal algorithms while being computation ally simpl er and requirin g less statistical knowledge. In particular , we resort to e vent-based online policies which react to a change i n fading level or an energy arri va l. Whene ver an e vent i s detected, the on line policy decides on a n e w po wer l e vel. Not e that the transmissio n is subject to av ailabili ty of energy and the E max constraint. 1) Const a nt W ater Level P olicy: The const ant water leve l p olicy makes o nline decisions for the t ransmit power whenev er a change in fading lev el is observed throu gh th e causal feedback. Assuming that the knowledge of the ave rage rechar ge rate P is a va ilable to the transmi tter and that fading density f h is known, the poli cy calculates h 0 that solves th e fol lowing equation . Z ∞ h 0  1 h 0 − 1 h  f h ( h ) dh = P (34) Whene ver a change in the fading level occurs, the p olicy decides on the following power level p i =  1 h 0 − 1 h i  + . If the energy in the battery is nonzero, t ransmission with p i is allowed, otherwise the transmit ter becomes sil ent. Note that t his po wer control poli cy i s the same as the capacity achieving power control poli cy in a stationary fading channel [16] with an av erage power constraint equal to the av erage rechar ge rate. In [2], this policy is prov ed to be stabili ty op timal in the sense that all data qu eues with stabilizable arriv al rates can be stabil ized by policies in this form where the power budget is P − ǫ for some ǫ > 0 su f ficientl y small . Howe ver , for the time constrain ed setting, this po licy is strictly su boptimal as wil l be verified in the nu merical result s section . This policy requires the transmitter to know the mean value of the energy arri va l process and the full s tatistics of the channel fading. A channel state inform ation (CSI) feedback is required from t he recei ver t o the transmitter at the tim es of events only . 18 2) Energy Ad a ptive W ater-F i lling: Another reduced complexity e vent-based policy is obtained by adapting the w ater le vel to the ener g y le vel in each ev ent . Again the fading statistics is assumed to be known. Wheneve r an ev ent occurs, t he po licy determines a new power le vel. In particul ar , the cutoff fade level h 0 is calculated at each energy arri val tim e as the solution of th e foll owing equation Z ∞ h 0  1 h 0 − 1 h  f ( h ) dh = E cur r ent (35) where E cur r ent is t he energy le vel at the time of t he e vent. Then, the transmissio n power lev el is determined sim ilarly as p i =  1 h 0 − 1 h  + . Thi s policy requires transmit ter to know th e fading statistics. A CSI feedback is required from the receiver t o t he transm itter at the times of changes in the channel state. 3) T ime-Ener gy A d aptive W a ter-F illin g: A variant of the energy adapt iv e water filling policy is obtained by adapting the power to the ener g y level and t he remaining time t o the deadline. The cutoff fade le vel h 0 is calculated at each energy arriv al ti me as t he solution of the following equation. Z ∞ h 0  1 h 0 − 1 h  f ( h ) dh = E cur r ent T − s i (36) Then, the transmiss ion power leve l i s determi ned as p i =  1 h 0 − 1 h  + . V I I . N U M E R I C A L R E S U L T S W e consider a fading addit iv e Gaussian channel with bandwidth W where the inst antaneous rate is r ( t ) = W log (1 + h ( t ) p ( t )) (37) h ( t ) i s the channel SNR, i.e., the actual channel g ain divided by the noi se power sp ectral density multipli ed by the bandwidth, and p ( t ) is the transmit power at time t . Bandwidth is chosen as W = 1 M Hz for th e simulatio ns. W e will examine the deadline constrained th roughput performances of the optimal o f flin e policy , opti mal online policy , and other p roposed sub-opti mal online policies. In particular , we 19 compare the o ptimal performance with t he proposed sub-opt imal on line policies which are based on waterfilling [16]. The proposed sub-optimal onl ine policies use th e fading distribution, and react only to the new ener gy arriv als and fading level changes. These event-based algorit hms require less feedback and less com putation, ho we ver , the fact that they react only to the changes in the fading leve l and new ener gy arri va ls is a short coming of th ese pol icies. Since the system i s deadline const rained, the policies need t o take the remaining time int o account yet the proposed policies do not do this optimally . W e wi ll simulate th ese policies under various dif ferent settings and we wil l observe that th e propo sed s ub-optimal policies may perform very well in some cases while not as well in so me ot hers. W e perform all simulations for 1000 randomly generated realizations o f the channel fade pattern and δ = 0 . 00 1 is taken for th e calculati on of the opti mal online policy . The rates of Poisson mark processes for energy arri val and channel fading λ e and λ f are assumed to be 1. The uni t of λ e is J / sec and t hat of λ f is 1 / sec. Hence, the mean value of th e density functi on f e is also the av erage rechar ge rate and the mean value of f h is the av erage fading le vel. The changes in the fading lev el occur relatively slowly with respect to th e sym bol duration. f e is s et as a non-negativ e uniform random variable with mean P , and as the ener gy arri val is assumed to be smaller than E max , we ha ve 2 P < E max . Selection of th e E max constraint is just for illus tration. In real l ife, sensors may ha ve batteries of E max on t he order of k J but th e battery feeds all circuits in the s ystem. Here, we assume a fictitious batt ery that carries ener gy for only communication purposes. Hence, E max on the order of 1 J will be considered. W e will examine differe nt f adi ng distrib u tions f h . In particular , Nakagami distribution with dif ferent shape parameter m wil l be considered. W e im plement the specified fading by sampli ng its probability density function with sufficiently large number of points . In order t o assess th e performance, we find an upper bound on the performances of the pol icies by first assuming that the channel fa ding lev els and ener gy arriv als in the [0 , T ] i nterva l ar e known non-causally , and t hat the total energy that will arri ve in [0 , T ] is av ailable at t he transmitter at time t = 0 . Then, for the water level p w that i s obt ained by spreading the total energy to the interval [0 , T ] , with the corresponding fading lev els , y ield the throughp ut T ub defined in the 20 following T ub = W T K X i =1 l i 1 2 log 1 + h i  p w − 1 h i  + ! (38) as an upper bound for the av erage throughput i n the [0 , T ] i nterva l; here l i denotes the duration of the fade level in the i th epoch. Even the offline optimal policy has a smaller aver age th roughput than T ub as the causality const raint does not allow energies to be spread e venly in to t he entire interval. W e start wit h examining t he av erage throughput of the system under Rayleigh fading with SNR = 0 d B and deadline T = 10 sec, E max = 10 J as depicted in Fig. 6. W e observe that time-energy adaptive waterfilling policy performs quite close to t he op timal o nline pol icy in th e low rechar ge rate regime. It can be a viable policy to spread the incoming ener gy when the rechar ge rate is low; howe ver , its performance saturates as t he recharge rate is increased. In this case the incoming ener gy canno t be easil y accomm odated and more and more ener gy is lost due to overflo ws. Similar trends can be found i n Fig. 7 under very low rechar ge rate re gime in the same setting with only difference bein g the battery capacity E max = 1 J . Next, we examine the setting wi th T = 10 sec, E max = 10 J under Nakagami fading of m = 3 (av erage SNR = 5 dB) and we observe si milar performances as i n the previous cases. As a comm on behavior i n these settings, energy adaptiv e water -filli ng performs poorer with respect to the constant water leve l and time-energy adaptive water -filling schemes. Finally , we examine the policies under d if ferent deadline constraints and present the plots for Nakagami fading distribution with m = 5 in Fig. 9. A remarkable result is t hat as the deadline is increased, stability optimal [2] constant water level poli cy approaches the optim al online policy . W e conclude t hat the t ime-awa reness of th e optimal onl ine policy h as less and less importance as the deadline constraint b ecomes l ooser . W e also o bserve th at t he throughput of t he ener g y-adaptiv e w aterfilling policy is roughl y a cons tant regardless of the deadli ne. Moreover , the time-energy adaptiv e poli cy performs worse as T is increased because energies are spread to very long i ntervals rendering the transmit power very s mall and hence energy accumulates in the battery . This leads to significant energy overfl ows since t he battery capacity is limit ed, and the performance degrades. 21 0 0.5 1 1.5 2 2.5 3 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 Avg. Recharge Rate (J/sec) Avg. Throughput (Mbits / sec) T ub Optimal Offline Optimal Online Time−Energy Adaptive WF Constant Water Level Energy Adaptive WF Fig. 6. Performances of the policies for various energy arriv al rates under unit-mean Rayleigh fading , T = 10 sec and E max = 10 J. 0 0.05 0.1 0.15 0.2 0.25 0.3 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 Avg. Recharge Rate (J/sec) Avg. Throughput (Mbits / sec) T ub Optimal Offline Optimal Online Time−Energy Adaptive WF Constant Water Level Energy Adaptive WF Fig. 7. P erformances of the policies for v arious average recharge rates under unit-mean Rayleigh fading, T = 10 sec and E max = 1 J. V I I I . C O N C L U S I O N S W e dev eloped optimal ener gy m anagement schemes for energy harv est ing systems operating in fading channels, with finite capacity rechar geable batteries. W e considered two related problems under of flin e k nowledge o f the events: maximizing the number of b its sent b y a deadli ne, and 22 0 0.5 1 1.5 2 2.5 3 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 Avg. Recharge Rate (J/sec) Avg. Throughput (Mbits / sec) T ub Optimal Offline Optimal Online Time−Energy Adaptive WF Constant Water Level Energy Adaptive WF Fig. 8. Performances of the policies for dif ferent ener gy recharge rates under Nakagami fading with m = 3 , T = 10 sec and E max = 10 J. 10 20 30 40 50 60 70 80 90 100 0.4 0.5 0.6 0.7 0.8 0.9 1 Deadline T(sec) Avg. Throughput (Mbits / sec) T ub Optimal Offline Optimal Online Time−Energy Adaptive WF Constant Water Level Energy Adaptive WF Fig. 9. Performances of the policies with respect to deadline T under Nakagami fading distribution wi th m = 5 and average recharge rate P = 0 . 5 J/sec and E max = 10 J. minimizi ng the ti me i t takes to send a given amount of data. W e solved the first problem using a directional water -filling approach. W e solved the second problem by mapping it to the first probl em via th e maximum departu re curve function. Fin ally , we solved for throughp ut optimal policy for the deadline constrained setting under online knowledge o f the e vents usin g 23 dynamic p rogramming in conti nuous t ime. Our num erical results show the performances of these algorithms under offline and onlin e knowledge. R E F E R E N C E S [1] J. Lei, R. Y ates, and L. Greenstein, “ A generic model for optimizing single-hop transmission policy of replenishable sensors, ” IEEE T rans. W ire less Commun. , vol. 8, pp. 547–55 1, February 2009. [2] V . Sharma, U. Mukherji, V . Joseph, and S . Gupta, “Optimal energy managemen t policies for energy harvesting sensor nodes, ” IEEE T rans. W ir eless Commun. , vol. 9, pp. 1326– 1336, April 2010. [3] M. Gatzianas, L. Georgiadis, and L . T assiulas, “Control of wireless networks with rechargea ble batteries, ” IEEE T rans. W ir eless Commun. , vol. 9, pp. 581–593, February 2010. [4] C. Ho and R. Zhang, “Optimal energy allocation for wireless communications powered by energ y harvesters, ” in IEEE ISIT , June 2010. [5] J. Y ang and S . Ulukus, “T ransmission completion time minimization in an energy harvesting system, ” in CISS , March 2010. [6] J. Y ang and S. Ul ukus, “Optimal packet scheduling in an energy harvesting communication system, ” IEEE T rans. Comm. , submitted June 2010. Also av ailable at [7] K. T utuncuoglu and A. Y ener , “Optimum transmission policies for battery limited energy harvesting nodes, ” IEEE T rans. W ir eless Comm. , submitted, S eptember 2010. Also av ailable at [arXiv:1010.628 0]. [8] M. Zafer and E. Modiano, “ A calculus approach to energy-ef ficient data transmission wi th quality-of-service constraints, ” IEEE/ACM T rans. on Networking , vol. 17, pp. 898–91 1, June 2009. [9] A. Fu, E. Modiano, and J. Tsitsiklis, “Optimal energy allocation and admission control for communications sattelites, ” IEEE/ACM T rans. on Networking , vol. 11, pp. 488–50 0, June 2003. [10] A. Fu, E. Modiano, and J. N. T sitsiklis, “Optimal transmission scheduling ov er a fading channel with energy and deadline constraints, ” IEEE T rans. W ireless Commun. , vol. 5, pp. 630–641, March 2006. [11] B. Prabhakar , E. Uysal-Biyikog lu, and A. El Gama l, “Energy-e fficient transmission o ver a wireless link via lazy scheduling, ” in IEEE INF OC OM , April 2001. [12] E. Uysal-Biyikoglu and A. El Gamal, “On adapti ve transmission for energy efficienc y in wireless data networks, ” I E EE T rans. Inform. Theory , vol. 50, pp. 3081–30 94, December 2004. [13] M. Zafer and E. Modiano, “Continuous t ime optimal rate control f or delay constrained data transmission, ” in Allerton Confer ence , September 2005. [14] D. P . Bert sekas, Dynamic Pr ogramming and Optimal Contr ol . Athena S cientific, 2007. [15] D. P . Bert sekas, Con vex Analysis and Optimization . Athena Scientific, 2003. [16] A. Goldsmith and P . V araiya, “Capacity of fading channe ls with channel side information, ” IEEE T rans. Inform. Theory , vol. 43, pp. 1986–1992 , Nov . 1997. 24