Privacy-Preserving Machine Learning Using EtC Images

Priv acy-Preserving Mac hine Learning Using EtC Images Ay ana Ka wam ura, Y uma Kinoshita, Hitoshi Kiy a T oky o Metrop olitan Universit y , T okyo, Japan ABSTRA CT In this pap er, w e propose a nov el priv acy-preserving machine learning scheme with encrypted images, called EtC (Encryption-then-Compression) images. Using mac hine learning algorithms in cloud environmen ts has b een spreading in many fields. Ho wev er, there are serious issues with it for end users, due to semi-trusted cloud pro viders. Accordingly , we propose using EtC images, which ha v e b een prop osed for EtC systems with JPEG compression. In this pap er, a nov el prop ert y of EtC images is considered under the use of z-score normalization. It is demonstrated that the use of EtC images allo ws us not only to protect visual information of images, but also to preserve b oth the Euclidean distance and the inner pro duct b et ween vectors. In addition, dimensionalit y reduction is shown to can b e applied to EtC images for fast and accurate matc hing. In an exp eriment, the prop osed sc heme is applied to a facial recognition algorithm with classifiers for confirming the effectiveness of the sc heme under the use of supp ort v ector machine (SVM) with the kernel trick. Keyw ords: Supp ort v ector machine, Encryption-then-Compression, Priv acy-preserving 1. INTR ODUCTION Cloud computing and edge computing ha ve been spreading in man y fields with the dev elopmen t of cloud services. Ho wev er, the computing en vironmen t has some serious issues for end users, such as the unauthorized use of services, data leaks, and priv acy being compromised due to unreliable pro viders and some acciden ts. 1 V arious metho ds hav e b een prop osed for priv acy-preserving computation. The metho ds are classified into tw o types: p erceptual encryption-based type 2–5 and homomorphic encryption (HE)-based one. 6–9 In recent years, considerable efforts hav e b een made in the fields of fully HE and multi-part y computation. 10 Some attempts with HE-based one hav e b een made on learning. 6–9 Ho wev er, HE-based schemes require algo- rithms sp ecialized for computing encrypted data be prepared, and high computational complexity . 7 In addition, it is also difficult to maintain high computational accuracy . 8 Because of this, w e prop ose a priv acy-preserving mac hine learning scheme based on a p erceptual encryption metho d in this pap er. W e focus on encrypted images, called ”encryption-then-compression (EtC) images,” whic h ha ve b een prop osed for EtC systems with JPEG compression. 2, 3, 11–14 So far, the safety of the EtC systems has b een ev aluated on the basis of the key space under the assumption of brute-force attacks, and robustness against jigsa w puzzle attacks has b een discussed. 15, 16 In this pap er, a no vel prop ert y of EtC images is considered under the use of z-score normalization. The nov el prop ert y allo ws us to securely compute machine learning algorithms without any degradation in p erformances. In addition, dimensionality reduction is shown to mak e it possible to apply EtC images for fast and accurate matc hing of visual features. In an exp erimen t, the proposed sc heme is applied to a facial recognition algorithm with classifiers to confirm the effectiveness of the scheme under the use of support vector machine (SVM). 2. ETC IMAGE As men tioned abov e, we fo cus on EtC images. EtC ones are robust against v arious ciphertext-only attacks, including jigsaw puzzle solver attac ks. 15, 16 In this paper, w e show that EtC images ha ve a nov el property , and this property allo ws us to carry out priv acy-preserving machine learning. In Figure 1, 2, 3 the pro cedure of generating EtC images is shown. In this pap er, images encrypted with the ab o ve steps are referred to as EtC images. I g I e K 1 K 3 K 2 Grayscale-based Image Generation Method Dividing into 8x8 blocks Block Scrambling Negative-Positive Transformation Block Rotation and Inversion Key Block Scrambling Steps I Color Space Transformation YCbCr I RGB Figure 1: Encryption steps for generating EtC images 2, 3 Figure 2: Priv acy-preserving machine learning 3. PR OPOSED PRIV A CY-PRESER VING MACHINE LEARNING 3.1 Scenario The procedure of the prop osed scheme is summarized here. A. Model T raining (See Fig.2): 1) Prepare EtC images by using a blo ck-based encryption method 2 in a client, and send them to a cloud serv er. 2) Perform dimensionality reduction in the encrypted domain, follow ed by training a model with the reduced data in the encrypted domain. B. T esting: 1) Send an EtC image as a test one to the cloud server. 2) Solve a classification problem with the trained mo del, after the dimensionality reduction of the image as well as for training ones. 3) Return a result to the clien t. Note that the cloud provider has no both visual information of images and the secret k ey . In addition, the pro vider can carry out directly mac hine learning algorithms with encrypted data. The performance of the prop osed sc heme will b e demonstrated to b e the same as that of plain images under the use of z-score normalization. 3.2 No vel prop erty of EtC images In the encryption steps in Fig.1, blo c k scramble, blo c k rotation and in v ersion are carried out for p ermuting pixels. These op erations can b e represen ted as an orthogonal matrix. As a result, blo c k scramble, blo c k rotation and inv ersion preserve both the Euclidian distance and the inner pro duct b etw een vectors. In addition, when applying z-score normalization to EtC images, its normalization allows us to main tain the inner pro duct b etw een v ectors ev en when an negativ e-p ositive transformation is carried out. Therefore, EtC images can provide exact the same as the performance of plain images under the use of t ypical mac hine learning algorithms with the k ernel tric k, including k-nearest neighbor, random forest, the radial basis function (RBF) kernel and the p olynomial k ernel. 4. SIMULA TION The prop osed scheme w as applied to facial recognition exp eriments that were carried out with a SVM algorithm. 4.1 Sim ulation condition W e used the Extended Y ale F ace Database B 17 , whic h consists of 2432 fron tal facial images with 192 × 160 pixels for 38 p ersons. 64 images for each p erson were divided into half randomly for training data samples and queries. 8 × 8 w as used as a blo c k size of encryption. Dimensionalit y was reduced with reduction ratios of 1/20, 1/40, 1/60, and 1/80, so 192 × 160 = 30720 dimensions was reduced to 1536 dimensions for a ratio of 1/20. After the dimensionalit y reduction, w e applied z-score normalization to the reduced vectors. Figure 3 sho ws an example of original images and the EtC images. SVM w as used as an example of mac hine le arning algorithms, with the RBF k ernel and linear kernel. (a) Original image (b) EtC image Figure 3: Example of facial images 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 FAR,FRR FAR encrypted FRR encrypted FAR not encrypted FRR not encrypted (a) Linear k ernel 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 FAR,FRR FAR not encrypted FAR encrypted FRR encrypted FRR not encrypted (b) RBF k ernel Figure 4: FRR and F AR for SVM ( B 1 = B 2 = . . . = B N , reduction ratio = 1/64) 4.2 Sim ulation results In this exp eriment, the false reject rate (FRR), false accept rate (F AR), and equal error rate (EER), at which F AR is equal to FRR, were used to ev aluate the p erformance. 4.2.1 Key condition 1: B 1 = B 2 = . . . = B N Exp erimen tal results are shown in Figure 4 in the case of using a common key for all clien ts (Key condition 1). The results demonstrate that SVM classifiers with encrypted images (encrypted in Fig.4) p erformed the same as SVM classifiers with the original images (not encrypted in Fig.4). F rom the results, the prop osed scheme w as confirmed to give no influence on the p erformance of SVM classifiers under k ey condition 1 and z-score normalization. 4.2.2 Key condition 2: B 1 6 = B 2 6 = . . . 6 = B N T ables 1 and 2 show results in the case of using a differen t key for eac h client (k ey condition 2). In this condition, it is exp ected that a query is authenticated only when it meets tw o requirements, i.e., the same key and the same p erson as for training the mo del, although only the same person is required under key condition 1. Therefore, the performance with encrypted images is slightly different from that of plain images, so the EER v alues of ”encrypted” in T ables 1 and 2 w ere smaller than those of ”not encrypted” due to the strict requirements. As a result, the EER v alues under key condition 2 outp erformed those under k ey condition 1. 5. CONCLUSION W e proposed a priv acy-preserving machine learning scheme with EtC images. A nov el property allows us to apply EtC images to some machine learning algorithms without any degradation in classification p erformance. Moreo ver, t wo key conditions were considered to enhance robustness against v arious attacks. A num b er of facial recognition experiments using SVM were also demonstrated to show the effectiveness of the prop osed sc heme. REFERENCES [1 ] Huang, C. T., Huang, L., Qin, Z., Y uan, H., Zhou, L., V aradhara jan, V., and Kuo, C.-C. J., “Survey on securing data storage in the cloud,” APSIP A T ransactions on Signal and Information Pro cessing 3(e7) (2014). T able 1: EER v alues for SVM with linear k ernel ( B 1 6 = B 2 6 = . . . 6 = B N ) reduction ratio not encrypted encrypted 1/20 0.0223 0.000744 1/40 0.0247 0.000835 1/60 0.0271 0.000777 1/80 0.0296 0.000779 T able 2: EER v alues for SVM with RBF kernel ( B 1 6 = B 2 6 = . . . 6 = B N ) reduction ratio not encrypted encrypted 1/20 0.0504 0.000448 1/40 0.0644 0.00112 1/60 0.0732 0.00779 1/80 0.0863 0.00855 [2 ] Chuman, T., Sirichotedumrong, W., and Kiya, H., “Encryption-then-compression systems using gra yscale- based image encryption for jp eg images,” IEEE T ransactions on Information F orensics and security 14(6), 1515–1525 (2019). [3 ] Sirichotedumrong, W. and Kiya, H., “Grayscale-based blo c k scrambling image encryption using ycbcr color space for encryption-then-compression systems,” APSIP A T ransacrions on Signal and Information Pro cess- ing 8 (e7 2019). [4 ] Itier, V., Puteaux, P ., and Puech, W., “Recompression of jp eg crypto-compressed images without a k ey ,” IEEE T ransactions on Circuits and Systems for Video T echnology , 1–1 (2019). [5 ] T anak a, M., “Learnable image encryption,” IEEE International Conference on Consumer Electronics- T aiwan (ICCE-TW) , 1–2 (2018). [6 ] Phong, L., Aono, Y., Ha yashi, T., W ang, L., and Moriai, S., “Priv acy-preserving deep learning via additiv ely homomorphic encryption,” IEEE T ransactions on Information F orensics and Security 13(5), 1333–1345 (2018). [7 ] W ang, Y., Lin, J., and W ang, Z., “An efficien t conv olution core architecture for priv acy-preserving deep learning,” Pro c. IEEE International Symp osium on Circuits and System (2018). [8 ] Y ang, H., Huang, Y., Y u, Y., Y ao, M., and Zhang, X., “Priv acy-preserving extraction of hog features based on in teger v ector homomorphic encryption,” International Conference on Information Securit y Practice and Exp erience , 102–117 (2017). [9 ] Saxe, A. M., Bansal, Y., Dap ello, J., Adv ani, M., Kolchinsky , A., T racey , B. D., and Cox, D. D., “On the information bottleneck theory of deep learning,” In ternational Conference on Learning Representations (2018). [10 ] Araki, T., Barak, A., F uruk a wa, J., Lic hter, T., Lindell, Y., Nof, A., Ohara, K., W atzman, A., and W einstein, O., “Optimized honest-ma jorit y mp c for malicious adv ersaries - breaking the 1 billion-gate p er second barrier,” IEEE Symposium on Security and Priv acy (SP) , 843–862 (2017). [11 ] Kurihara, K., Shiota, S., and Kiya, H., “An encryption-then-compression system for jp eg standard,” Picture Co ding Symp osium (PCS) , 119–123 (2015). [12 ] Kurihara, K., Kikuchi, M., Imaizumi, S., Shiota, S., and Kiya, H., “An encryption-then-compression system for jp eg/motion jpeg standard,” IEICE T ransactions on F undamentals of Electronics, Comm unications and Computer Sciences 98(11), 2238–2245 (2015). [13 ] Kurihara, K., Imaizumi, S., Shiota, S., and Kiya, H., “An encryption-then-compression system for lossless image compression standards,” IEICE transactions on information and systems E100-D(1), 52–56 (2017). [14 ] W atanab e, O., Uchida, A., F ukuhara, T., and Kiy a, H., “An encryption-then-compression system for jp eg 2000 standard,” IEEE International Conference on Acoustics, Sp eech and Signal Processing (ICASSP) , 1226–1230 (2015). [15 ] Chuman, T., Kurihara, K., and Kiya, H., “On the security of blo ck scrambling-based etc systems against jigsa w puzzle solver attacks,” IEEE In ternational Conference on Acoustics, Speech and Signal Pro cessing (ICASSP) , 2157–2161 (2017). [16 ] Sirichotedumrong, W., Ch uman, T., Imaizumi, S., and Kiya, H., “Gra yscale-based block scrambling im- age encryption for so cial netw orking services,” Pro c. IEEE In ternational Conference on Multimedia and Exp o(ICME) (2018). [17 ] Georghiades, A., Belh umeur, P ., and Kriegman, D., “F rom few to many: Illumination cone mo dels for face recognition under v ariable lighting and pose,” IEEE T rans. Pattern Analysis and Mac hine Intelligence 23(6), 643–660 (2001).