Kubernetes Virtual Warehouse Placement based on Reinforcement Learning

doi:10.23940/ijpe.21.07.p2.579588

Abstract

Abstract: As a method to build and run applications, cloud native can make frequent and predictable major changes to the system, is closely correlated to the fast iteration and automated deployment, and is adaptive to the network era with high-speed changes in large amounts of data. Nevertheless, cloud native is still growing and there are still many problems to be solved. This paper selected Kubernetes, the cornerstone of the cloud native ecosystem, and Docker, the huge orchestration system that manages containers, to deploy a Virtual Warehouse for managing mirror resources. With the rapid development of artificial intelligence (AI), the reinforcement learning (RL) algorithm has been extensively applied in AI by virtue of two features: trial-and-error and the value of long-term rewards. The RL algorithm can be applied to multiple problem scenarios. The study object in this paper meets the requirements of the RL algorithm and has constantly updated environmental conditions. In consideration of the implementation of automated container operations by k8s, it was proposed to learn in the existing environment through RL and with changes in demand as the environment updates, until the Virtual Warehouse converges to the optimum location. In this paper, simulation modeling was made on the cloud native process. The model data were trained according to the model environment and by the RL algorithm to obtain the optimum warehouse placement. The warehouse location parameters obtained were substituted into the simulation environment. In addition, the abstract task class was pulled according to the extended image to obtain the delay time of different tasks, thus verifying the superiority of the RL algorithm in the K8S warehouse placement.

Key words: Cloud native, CloudSim, reinforcement learning, Kubernetes, virtual warehouse placement

Haoran Li, Dongcheng Li, W. Eric. Wong, Deze Zeng, and Man Zhao. Kubernetes Virtual Warehouse Placement based on Reinforcement Learning [J]. Int J Performability Eng, 2021, 17(7): 579-588.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

1. Gannon, D., Barga, R. and Sundaresan, N.Cloud-native applications.IEEE Cloud Computing, 4(5), pp.16-21, 2017.
2. Arundel, J. and Domingus, J.Cloud Native DevOps with Kubernetes: building, deploying, and scaling modern applications in the Cloud.O'Reilly Media, 2019.
3. Li X., Jiang X., Huang P. and Ye K.DartCSim: An enhanced user-friendly cloud simulation system based on CloudSim with better performance. In 2012 IEEE 2nd International Conference on Cloud Computing and Intelligence Systems (Vol. 1, pp. 392-396). IEEE, 2012.
4. Jiang W., Feng G., Qin S., Yum T.S.P. and Cao, G. Multi-agent reinforcement learning for efficient content caching in mobile D2D networks.IEEE Transactions on Wireless Communications, 18(3), pp.1610-1622, 2019.
5. Kim J., Lee J., Kim T. and Pack S.Deep Reinforcement Learning based Cloud-native Network Function Placement in Private 5G Networks. In 2020 IEEE Globecom Workshops (GC Wkshps (pp. 1-6). IEEE, 2020.
6. Talpaert V., Sobh I., Kiran B.R., Mannion P., Yogamani S., El-Sallab, A. and Perez, P. Exploring applications of deep reinforcement learning for real-world autonomous driving systems.arXiv preprint arXiv:1901.01536, 2019.
7. Rizk, Y., Awad, M. and Tunstel, E.W.Decision making in multiagent systems: A survey.IEEE Transactions on Cognitive and Developmental Systems, 10(3), pp.514-529, 2018.
8. Fan J., Wang Z., Xie Y. and Yang Z.A theoretical analysis of deep Q-learning. In Learning for Dynamics and Control (pp. 486-489). PMLR, 2020.
9. Mao Y., Fu Y., Gu S., Vhaduri S., Cheng L. and Liu Q.Resource management schemes for cloud-native platforms with computing containers of docker and kubernetes.arXiv preprint arXiv:2010.10350, 2020.
10. Moravcik, M. and Kontsek, M.Overview of Docker container orchestration tools. In 2020 18th International Conference on Emerging eLearning Technologies and Applications (ICETA) (pp. 475-480). IEEE, 2020.
11. McKendrick, R. Mastering Docker: Enhance your containerization and DevOps skills to deliver production-ready applications. Packt Publishing Ltd., 2020.
12. Oladimeji O.O., Oyeyiola D., Oladimeji O. and Oyeyiola P.A Comprehensive Survey On Cloud Computing Simulators.Scientific Journal of Informatics, 8(1), pp.51-59, 2021.
13. Hussain A., Aleem M., Iqbal M.A. and Islam M.A.Investigation of cloud scheduling algorithms for resource utilization using cloudsim.Computing and Informatics, 38(3), pp.525-554, 2019.
14. Rani, E. and Kaur, H.Study on fundamental usage of CloudSim simulator and algorithms of resource allocation in cloud computing. In 2017 8th International Conference on Computing, Communication and Networking Technologies (ICCCNT) (pp. 1-7). IEEE, 2017.
15. Piraghaj S.F., Dastjerdi A.V., Calheiros R.N. and Buyya R.ContainerCloudSim: An environment for modeling and simulation of containers in cloud data centers.Software: Practice and Experience, 47(4), pp.505-521, 2017.

[1]	Shaili Mishra and Anuja Arora. Double Deep Q Network with Huber Reward Function for Cart-Pole Balancing Problem [J]. Int J Performability Eng, 2022, 18(9): 644-653.
[2]	Yuting Cheng, Dongcheng Li, W. Eric Wong, Man Zhao, and Dengfeng Mo. Multi-UAV Collaborative Path Planning using Hierarchical Reinforcement Learning and Simulated Annealing [J]. Int J Performability Eng, 2022, 18(7): 463-474.
[3]	Chengjie Xu, Dongcheng Li, W. Eric Wong, and Man Zhao. Service Caching Strategy based on Edge Computing and Reinforcement Learning [J]. Int J Performability Eng, 2022, 18(5): 350-358.
[4]	Shujun Pei, Qinggen Zhang, and Xuehui Cheng. Workflow Scheduling using Graph Segmentation and Reinforcement Learning [J]. Int J Performability Eng, 2020, 16(8): 1262-1270.