A Novel Dynamic Peer-to-Peer Clustering Algorithm and Its Application to Aggregate Energy Storage Systems

1 A No v el Dynamic Peer -to-Peer Clustering Algorithm and Its Application to Aggre gate Ener gy Storage Systems Runfan Zhang, Branislav Hredzak, Senior Member , IEEE Abstract —The proposed distributed dynamic clustering algo- rithm enables to group agents based on their pre-selected feature states. The clusters are determined by comparing the distance of the agents’ current feature states with av erage estimates of the states in all clusters. The algorithm also pro vides average estimates of pre-selected auxiliary states that can be utilized f or control purposes. T wo example applications of the algorithm ar e introduced. In the first application, the algorithm is applied to a microgrid with distributed batteries that are contr olled to achieve a common state of charge within a group. Howev er , a random selection of the batteries’ gr oups results in additional power losses during operation. The algorithm reduces the power losses by clustering the batteries based on the selected feature states: local loads and battery capacities, while the state of charges and output voltages are selected as auxiliary states f or control purposes. In the second application, the algorithm is used to form a virtual energy storage from batteries distributed in a micr ogrid. Index T erms —Distributed clustering, multi-agent, cluster , mi- crogrid, battery energy storage, state of charge balancing, power loss reduction, voltage restoration. I . I N T RO D U C T I O N C LUSTERING is a division of data into groups with similar properties [1]. V arious centralized clustering al- gorithms hav e been researched, such as k-means [2], hierar- chical clustering [3], self-organization map [4] or e xpectation maximization clustering algorithms [5]. Besides, distributed clustering algorithm have been proposed for distributed data en vironment [6]. Howe ver , although the data is distributed, these algorithms require a main site for global clustering, with each data storage communicating with the main site through a centralized network [7]. In addition, all before-mentioned algorithms are clustering based on previously collected data and not on real-time system states. Therefore, these algorithms are inadequate for clustering multi-agent based systems com- municating via a distributed netw ork. Microgrids are small-scale power networks that supply local loads in small geographical areas. The microgrid enables pene- tration of rene wable energy sources, such as photov oltaic (PV) and wind generation, and energy storage [8]. T o control the microgrid, the traditional hierarchical power systems control architecture can be applied [9]: primary , secondary and tertiary control le vels. The primary control level applies droop control to provide proportional power sharing. Then, the secondary Runfan Zhang and Branislav Hredzak are with the School of Electri- cal Engineering and T elecommunications, Uni versity of New South W ales, Sydney , NSW 2052 Australia (email: runfan.zhang@student.unsw .edu.au; b .hredzak@unsw .edu.au;). control eliminates the voltage and/or frequency deviations caused by the droop control and, at the same time, provides accurate power sharing. Po wer flow objecti ves objecti ves are realized at the tertiary control level. T o impro ve the power quality and reliability in microgrids, a state of charge (SoC) balancing control strategy at the secondary control level was proposed for distributed energy storages (ESs) [9]. Not only the strategy maintains the SoCs of all ESs at the same lev el, but it can also provide a proportional power sharing between ESs after a balanced SoC is achiev ed. Howe ver , for a microgrid with a large number of heterogeneous ESs and different local loads, maintaining all ESs SoCs at the same le vel is not alw ays the best option. This is because during and after the SoC balancing, the power network line currents inevitably result in power losses. Literature on application of clustering algorithms in micro- grids can be broadly di vided into three categories. In the first category , po wer systems are clustered based on previous data profiles, such as 24 hour data profiles of loads and renewable energy generation [10]–[12]. The purpose of clustering is to achiev e economical operation. The second category is based on stochastic modeling utilizing a one year data profile (365 days) and dividing the data into different probabilistic scenarios to deal with uncertainties [13], [14]. The last one defines a set of power systems in a geographical area without using any clustering algorithm [15]–[17]. Besides, clustering methods were used for partitioning a central network based on voltage source con verter sensiti vity matrices, voltage influence factors and merge matrices [18], and by finding a critical load [19]. Howe ver , all abov e clustering algorithms require past data and are run off-line, i.e. the clustering is determined before the system operation. Motiv ated by the above discussion, this paper proposes a novel, fully distrib uted dynamic on-line clustering algo- rithm based on dynamic states of multi-agents communicating through a distributed communication network. The clustering algorithm dynamically aggregated ESs into clusters with se- lected similar features and maintains a balanced SoC within each cluster . The proposed algorithm is applied (i) to mitigate power losses associated with the traditional SoC balancing of fixed ES clusters distributed in a microgrid and (ii) to form a virtual energy storage . The salient nov el features of this paper are: 1) The clustering algorithm is fully distributed and can be implemented with any existed distributed control method. 2 2) The clustering is done on-line, i.e. the clustering can dynamically adjust to v ariation in multi-agent states and does not require previously collected data. 3) The proposed algorithm is applied to a microgrid (i) to reduce power losses associated with SoC balancing of batteries in fix ed clusters and (ii) to form a virtual ener gy storage. 4) The clustering algorithm is robust to multi-agents’ state variation. As a result, frequent cluster changes, caused by fast state variations, can be eliminated. The rest of this paper is org anized as follo ws. Section II presents the proposed algorithm. Implementation of the algorithm to a microgrid with distributed batteries is discussed in Section III. Section IV concludes the paper . I I . D I S T R I B U T E D C L U S T E R I N G A L G O R I T H M This section introduces the proposed distributed clustering algorithm for multi-agents. The clustering problem can be defined as: cluster N agents into M groups based on some selected features propagated through a distributed network. Also assume that the estimations of average states inside of a cluster can be obtained by each agent through the distrib uted network. Next, the concept of a distributed network is introduced and then the proposed algorithm is presented. A. Distributed Communication Network Multi-agents communicate with neighbours through a net- work represented by a sparse graph G ( V , E ) , with nodes V = { 1 , · · · , N } and edges E [20]. Each graph node represents an agent. Elements of E are denoted as ( i, j ) , where ( i, j ) ∈ E if there is a link allowing information to flow from node i to node j . The neighbours of node i are gi ven by N i , where j ∈ N i , if ( i, j ) ∈ E . The graph adjacency matrix is given by A = [ a ij ] ∈ R N × N , a ij =  α, ( j, i ) ∈ E 0 , other wise , (1) where α is the coupling strength. Then, the graph Laplacian matrix is L = D − A , where D = diag { d i } , and d i = P N j =1 a ij is the in-degree of the communication netw ork. B. Dynamic Clustering Algorithm The feature states of an agent, used to determine the clusters, are defined as x i ∈ < n . The proposed algorithm dynamically clusters N agents into M clusters based on the feature states. Estimates of the average of the states x i in the j -th cluster are ¯ x cstrj i , where i = 1 , · · · , N and j = 1 , · · · , M . Each i -th agent has continuous access to the latest average states ¯ x cstrj i of all clusters through the communication network. First, select (arbitrary) initial estimates of ¯ x cstrj i . Then, in order to decide/update to which cluster the i -th agent belongs, its current state x i is compared with all current estimates of ¯ x cstrj i , j = 1 , · · · , M . The k -th estimate ¯ x cstrk i which has the smallest distance from the current state x i , as giv en by (4), denotes that the i -th agent is allocated to the k - th cluster . Once all clusters are determined, each i -th agent cst r j l x li a 1 s i j i cst r x + − + + 1 cs t r j l x + li a 1 s i 1 cs t i rj x + + − + + ( ) 1 , , a r g m in , 1 , , cst r k cst r j i i i kM x x x j M = = − = 1/ i Σ i x i x 1/ i Σ , 1 , , c s t r i j j x M = i f j k = i f j k  k k * , c s t r i o t h er es t i m a x t i o n s ( ) __ k cls t r r s l t a ( ) __ cl s t r r s l t b Fig. 1. Distributed clustering algorithm for the i -th agent. The double line arrows represent distributed communication. N ode 1 N ode 2 N ode 3 N ode 4 N ode 5 N ode 6 N ode 7 N ode 8 N ode 9 Fig. 2. Illustration of the proposed clustering algorithm. Nine agent nodes are clustered into three groups. The black lines represent the distributed communication network between agent nodes. The colored dash lines indicate virtual communications within a cluster . One color represents one cluster . estimates the av erage of the states of the k -th cluster to which it belongs based on its current state x i and the k -th cluster agents’ estimates passed through the network, as gi ven by (2). Neighbours’ estimates that do not belong to the k -th cluster , j ∈ { 1 , · · · , M } , j 6 = k , are passed through, as given by (3). ˙ ¯ x cstrk i = ˙ x i + X l ∈N i a il  ¯ x cstrk l − ¯ x cstrk i  , (2) ˙ ¯ x cstrj i = 1 |N i | X l ∈N i ˙ ¯ x cstrj l + X l ∈N i a il  ¯ x cstrj l − ¯ x cstrj i  , j 6 = k , (3) where |N i | denotes the number of neighbours of the i -th agent. 3 x i ∈ ( k -th cluster      ¯ x cstrk i = arg min k ∈{ 1 , ··· , M }     x i − ¯ x cstrj i     , j = 1 , · · · , M ) . (4) Each i -th agent has always access to two results: 1) cl str r slt a : information to which k -th cluster it be- longs; 2) cl str r slt b : estimates of the av erage of the feature states ¯ x cstrj i in the j -th cluster , j = 1 , · · · , M . The algorithm is illustrated in Fig. 1 and can be summarized as Algorithm 1 . Algorithm 1 Distributed Dynamic Clustering 1: design G ( V , E ) and M 2: initialize ¯ x cstrj i , j = 1 , · · · , M 3: each i -th agent at time t 4: measure x i 5: receiv e ¯ x cstrj l , l ∈ N i , j = 1 , · · · , M from neigh- bours 6: k ← label of the cluster with the smallest distance between x i and ¯ x cstrj i , j = 1 , · · · , M , where k ∈ { 1 , · · · , M } 7: if j = k then 8: Eq. (2) 9: else 10: Eq. (3) 11: end if 12: send ¯ x cstrj i , j = 1 , · · · , M to neighbours 13: end each 14: Each local agent i implements the same algorithm. Example : T o further clarify the algorithm, consider the communication network structure in Fig. 2. In the figure, nine agent nodes are clustered into three groups. In each cluster , the av erage estimations are calculated based on the agents in the group, i.e. only the agents in the same cluster communicate with each other to obtain the average estimates through virtual communication lines shown as colored dash lines. The agents which are not in the cluster simply pass trough estimates of the other clusters to its neighbours. As- sume the red dashed line connecting nodes 1, 5 and 9 denote the k -th cluster . Nodes 1, 5, and 9 obtain the average estimate of the cluster ¯ x cstrk i using (2). Howe ver , for the other two clusters’ estimates ¯ x cstrj i , j 6 = k , nodes 1, 5, and 9 apply (3) to estimate the averages. Similarly , the other six agents ( i 6 = 1 , 5 , and 9 ) obtain the average estimates of the red line cluster k , ¯ x cstrk i , i 6 = 1 , 5 , and 9 , by applying (3). As a result, the nodes 1, 5, and 9 in the red line cluster k directly communicate with each other through the virtual red color line links. The other agents just transmit the estimations ¯ x cstrk i , i = 1 , 5 , and 9 . C. Utilization of Clusters In some cases, other states than the feature states may be required for control, optimization and analysis within the cst r j l z li a 1 s i j i cst r z + − + + 1 cs t r j l z + li a 1 s i 1 cs t i rj z + + − + + 1/ i Σ i z i z 1/ i Σ , 1 , , c s t r j i z j M = k , f r om c l us t e r i ng al gor i t hm i f j k = i f j k  ( ) __ cl s t r r s l t c ( ) __ c ls tr rs lt a Fig. 3. The average estimations of the auxiliary state z i for the i -th agent. The double line arrows represent distributed communication. cluster or between the clusters. These states are defined as the auxiliary states, z i ∈ < m , and are only used to calculate av erage of the states within a cluster . For example, in a microgrid, local loads and battery sizes are selected as feature states to determine clusters, whereas the SoCs are selected as auxiliary states for the av erage estimates of SoCs and the SoC balancing control within a cluster . Hence, similar to feature states, the estimates of the aux- iliary states z i are ¯ z cstrj i , where i = 1 , · · · , N and j = 1 , · · · , M . The estimates of the average states of the i -th agent in the j -th cluster can be obtained using the clstr rsl t a result and following the same procedure as for the estimates ¯ x cstrj i , ˙ ¯ z cstrk i = ˙ z i + X l ∈N i a il  ¯ z cstrk l − ¯ z cstrk i  , (5) ˙ ¯ z cstrj i = 1 |N i | X l ∈N i ˙ ¯ z cstrj l + X l ∈ N i a il  ¯ z cstrj l − ¯ z cstrj i  , j 6 = k . (6) Hence, each i -th agent has always access to cl str r slt c re- sult: estimates of the av erage of the auxiliary states ¯ z cstrj i , j = 1 , · · · , M . The algorithm is illustrated in Fig. 3. Including the auxiliary states into the Algorithm 1 giv es Algorithm 2 . 4 Algorithm 2 Distributed Dynamic Clustering with Utilization of Clusters 1: design G ( V , E ) and M 2: initialize ¯ x cstrj i , j = 1 , · · · , M 3: each i -th agent at time t 4: measure x i and z i 5: receiv e ¯ x cstrj l , l ∈ N i , j = 1 , · · · , M from neigh- bours 6: receiv e ¯ z cstrj l , l ∈ N i , j = 1 , · · · , M from neigh- bours 7: k ← label of the cluster with the smallest distance between x i and ¯ x cstrj i , j = 1 , · · · , M , where k ∈ { 1 , · · · , M } 8: if j = k then 9: Eq. (2) and (5) 10: else 11: Eq. (3) and (6) 12: end if 13: send ¯ x cstrj i , j = 1 , · · · , M to neighbours 14: send ¯ z cstrj i , j = 1 , · · · , M to neighbours 15: end each 16: Each local agent i implements the same algorithm. I I I . A P P L I C A T I O N S O F A L G O R I T H M T O C L U S T E R B AT T E R I ES I N M I C R O G R I D This section introduces two example applications of the pro- posed algorithm to cluster batteries distributed in a microgrid in order (i) to reduce the power losses and (ii) to form a virtual energy storage. A battery energy storage system (BESS) includes a battery , a bidirectional DC-DC con verter interfacing the battery with the microgrid and a local load. A. P ower Loss Reduction Balancing SoC of batteries distributed in a microgrid can improv e stability and reliability of the power system by making full use of energy stored in batteries and lengthening the battery lifetime [9]. Ho wev er , if the batteries are randomly clustered into fixed groups, it can result in power losses ev en though their SOCs are balanced. The additional power losses are caused by dif ferent local loads and different battery capacities, as explained next. 1) Local Loads: Assume that all batteries hav e the same capacity . The power losses, after a balanced SoC has been achiev ed, can be reduced by clustering the battery systems with similar local loads. This is because if each battery in a cluster has the same load, there will be no line currents flo wing between the batteries. 2) Battery Capacity: Assume that all batteries hav e equal local loads. After a balanced SoC has been reached, larger capacity batteries have to provide more power than smaller capacity batteries to maintain the balanced SoC. This results in additional line currents and hence power losses. Therefore, clustering batteries with similar capacity can reduce the po wer losses. Based on the above discussion, two feature states: local load and battery capacity are selected to cluster the batteries using the proposed distributed clustering algorithm and hence minimize the po wer losses. B. V irtual Ener gy Stora ge In this application, the batteries are clustered into virtual energy storage(s) to sell po wer to the main grid or to absorb the power from rene wable energy sources. It is assumed that the utility company provides required energy storage capacity (virtual energy storage) and its price to be formed from distributed BESSs. At the same time, all prosumers decide how much of their battery capacity will be made av ailable to form the virtual energy storage and at what price. Then, the utility company pins (broadcasts) its virtual energy storage requirements to at least one prosumer in an area where prosumers can exchange information through a neighbour-to-neighbour distrib uted communication network. The requirements are the initial values for the clusters. T wo feature states, price and battery capacity , are applied in the algorithm. The algorithm finds the clusters of batteries that are close to the feature states’ av erage estimations. All prosumers in a cluster can sell their available battery capacity at the av erage price. Subsequently , the company receives all infor- mation from the pining prosumers and selects clusters with sufficient energy capacity and at acceptable price to form the virtual energy storage. The SoC balancing control algorithm maintains the same SoC lev el in all batteries in the virtual energy storage by ensuring that batteries provide power in proportion to their own energy capacity . It should be noted that the selected feature states are not restricted to the price and energy capacity only , additional feature states, such as distance or geographical location, can be added to form virtual energy storage(s). I V . C O N C L U S I O N This paper proposed a distributed dynamic clustering al- gorithm for clustering agents based on pre-selected feature states. The clustering algorithm was applied (i) to cluster batteries with a similar load and energy capacity to reduce the po wer losses and (ii) to form a virtual energy storage based on price and energy capacity . The proposed algorithm can be applied to any other systems which require dynamic, on-line clustering based on pre-selected feature states, while using auxiliary states for any required control, optimization or analysis purposes. A C K N O W L E D G M E N T R. Zhang would like to gratefully thank for a scholarship from the China Scholarship Council. R E F E R E N C E S [1] J. Han, M. Kamber , and J. Pei, Data Mining: Concepts and T echniques . W altham: Morgan Kaufmann Publishers, 2001. [2] S.-S. Y u, S.-W . Chu, C.-M. W ang, Y .-K. Chan, and T .-C. Chang, “T wo improved k-means algorithms, ” Applied Soft Computing , vol. 68, pp. 747–755, 2018. [3] A. B. Geva, “Hierarchical unsupervised fuzzy clustering, ” IEEE Tr ans- actions on Fuzzy Systems , vol. 7, no. 6, pp. 723–733, Dec 1999. 5 [4] J. A. Flanagan, “Self-organisation in kohonen’ s som, ” Neural networks , vol. 9, no. 7, pp. 1185–1197, 1996. [5] M. Aci and M. A vci, “K nearest neighbor reinforced expectation max- imization method, ” Expert Systems with Applications , vol. 38, no. 10, pp. 12 585–12 591, 2011. [6] K. Thangav el, J. Kacprzyk, A. Abraham, A.-E. Hassanien, A. P . de Leon F de Carvalho, and V . Snel, Distributed Data Clustering: A Comparative Analysis , ser . Foundations of Computational, IntelligenceV olume 6. Berlin, Heidelberg: Springer Berlin Heidelberg, 2009, vol. 206. [7] M. Hai, S. Zhang, L. Zhu, and Y . W ang, “ A survey of distributed clustering algorithms, ” in 2012 International Conference on Industrial Contr ol and Electr onics Engineering , Aug 2012, pp. 1142–1145. [8] A. Bidram, V . Nasirian, A. Davoudi, and F . L. Lewis, Cooperative Synchr onization in Distributed Microgrid Contr ol . Springer, 2017. [9] T . Morstyn, B. Hredzak, and V . G. Agelidis, “Control strategies for microgrids with distributed energy storage systems: An overvie w , ” IEEE T ransactions on Smart Grid , vol. 9, no. 4, pp. 3652–3666, July 2018. [10] K. Li, J. Che, B. W ang, J. Zhang, F . W ang, and Z. Mi, “ A meta- heuristic optimization based residential load pattern clustering approach using improv ed gravitational search algorithm, ” in 2018 IEEE P ower Ener gy Society Innovative Smart Grid T echnologies Confer ence (ISGT) , Feb 2018, pp. 1–5. [11] H. Ryu, Y . Jung, and J. Park, “Energy storage control based on user clustering and battery capacity allocation, ” in 2017 IEEE P ower Ener gy Society General Meeting , July 2017, pp. 1–5. [12] H. Kim and C. Singh, “Three dimensional clustering in wind farms with storage for reliability analysis, ” in 2013 IEEE Grenoble Confer ence , June 2013, pp. 1–6. [13] D. Thomas, O. Deblecker, K. Genikomsakis, and C. S. Ioakimidis, “Smart house operation under pv and load demand uncertainty consider- ing e v and storage utilization, ” in IECON 2017 - 43rd Annual Confer ence of the IEEE Industrial Electr onics Society , Oct 2017, pp. 3644–3649. [14] C. Y ao, M. Chen, and Y . Hong, “Novel adaptive multi-clustering algorithm-based optimal ess sizing in ship power system considering uncertainty , ” IEEE T ransactions on P ower Systems , vol. 33, no. 1, pp. 307–316, Jan 2018. [15] M. He and M. Giesselmann, “Reliability-constrained self-organization and energy management towards a resilient microgrid cluster , ” in 2015 IEEE P ower Energy Society Innovative Smart Grid T echnologies Con- fer ence (ISGT) , Feb 2015, pp. 1–5. [16] J. Schlund, D. Steber , P . Bazan, and R. German, “Increasing the efficienc y of a virtual battery storage providing frequency containment reserve power by applying a clustering algorithm, ” in 2017 IEEE Innovative Smart Grid T echnologies - Asia (ISGT -Asia) , Dec 2017, pp. 1–8. [17] K. Shi, C. Gao, J. Li, J. Fan, and G. Cui, “The establishment of energy storage model and its application in secondary frequency regulation of air conditioning load, ” in 2017 4th International Confer ence on Systems and Informatics (ICSAI) , Nov 2017, pp. 363–368. [18] M. Bahramipanah, M. Nick, R. Cherkaoui, and M. Paolone, “Network clustering for voltage control in activ e distrib ution network including energy storage systems, ” in 2015 IEEE P ower Energy Society Innovative Smart Grid T echnolo gies Confer ence (ISGT) , Feb 2015, pp. 1–5. [19] D. D. Sharma, S. N. Singh, B. S. Rajpurohit, and F . G. Longatt, “Critical load profile estimation for sizing of battery storage system, ” in 2015 IEEE P ower Ener gy Society General Meeting , July 2015, pp. 1–5. [20] F . L. Lewis, H. Zhang, K. Hengster-Movric, and A. Das, Cooperative Contr ol of Multi-Agent Systems . London: Springer-V erlag, 2014.